MIS716

Artificial Intelligence for Business

Module 2: With Great Power Comes Great Responsibility

New Frontier
Technologies
(Power)

Responsible AI
(RAI)

AI Risks
(AIR)

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 2
Topic 6

AIR & RAI

recaps

A2
Consultation

Explainable
AI

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 3
Exponential Technologies Enabling the New Frontier

AI Risks
& Responsible AI
recaps

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 4
 AI Risks and Responsible AI Frameworks

External stakeholders Company

RAI
Performance
Ethics and Regulation
Ethical Security
Performance
AIR • Robustness and Security
• Bias and Fairness
• Interpretability and Expandability
Societal Control
Governance of AI
Economic

Internal stakeholders Society

Adapted from PwC
Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 5
AskMili’s Risks

Patient
• Possible skin Ethical
• Incorrect glucose infection
level Security
• Patient share Economic
• Incorrect advice device • Right to
• Privacy • Device loss device
breach • Affordability data
Society
Performance
Control

AskMili
• Fails to Economic

Vendor beam/read
accurately
Security • Lack of
governance
Ethical

• Incorrect patient (Commercial • Monetise

• Carer
info affect advice • Patient data vs health data
access to
integrity /loss interests) • Liable for Society data
Performance patient
health risks
Control

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 6
 Reframing ISO 38500 for Digital Evolution
MIS732, Mark Toomey, 2019

Source of • Governance of Digital Evolution is the

authority system by which an organisation directs

The System for Governance of Digital Evolution

and controls its evolving purpose and
Governing Body capability in an increasingly digital market.
for Digital Evolution • Business capability evolution exploits
Evaluate advances in technology and responds to
evolving use of technology.
Direct Monitor
• Business capability evolution drives digital
Digital Era Capability
Digital Evolution ,

Plans, Initiatives
Strategy, Policy,
Plans & Policy

transformation of the organisation and

enables the organisation’s participation in
digital transformation of the marketplace.

Conformance
Performance
Digital era
• Governance of Business Capability &
Evolution is the system by which current
and future (evolution) of business
capability is directed and controlled.
Digital era
Digital Evolution
Business
• Governance of Business Capability &
Initiatives Evolution finally puts Governance of IT into
Operations
its proper context – enabling the business!
Management of Digital Business Evolution - Digital Leadership
Slide 7
Deakin University CRICOS Provider Code: 00113B
Designing Governance Arrangements
MIS732, Mark Toomey, 2019

Governance
Business Capability Governance
of IT enabled Health Committee
Service capability
Governance Governance
Individual Capability capability Individual Capability
Manage and
of ITdeliver
enabled (the evolving
Hospital IT
Management
Governance Team
enabled capabilities of) an effective health
of IT enabled Patientservice
Treatment capability
Governance Team
Hospital Management Asset Management
Patient Treatment
Building Management Asset Tracking Other Treatments Patient Surgery

Inventory Planning
Adapted from a Capability based Architecture model

Catering Management Patient Exercise Patient Radiation

Maintenance Management
PhysiotherapyGovernance
developed by Justin Butcher at MXA Consulting

Individual Capability
Financial Management of IT enabled Patient Rehabilitation
Governance Team capability
Human Resource Management
Patient Rehabilitation
Infection Governance
Management Recruitment
Rostering Governance
Individual Capability
of IT enabled
Individual Capability
of IT enabled Medicine Management Patient Exercise
Governance Team
Client Pathology capability Governance Team
Client Triage capability

Client Pathology Client Triage

Patient Visits and Patient Monitoring
Specimen Collection Client Assessment
Consultation
Specimen Measurement
Client Referral
Results Analysis Citizen Education Medical Research

Results Reporting Client Triage Reporting

Slide 8
Deakin University CRICOS Provider Code: 00113B
Timer:
Self-driving car fleet – a new capability - a disruption
Strategy, industry
requirements and
internal policies

• Governance structure
• A representative in the overall steering committee
• A specific committee to direct and control the capability planning, building Planning and
Ecosystem: program
oversight, insurance,
and running of the new fleet sourcing, change
management
• How does AI align with the business strategy?
• Innovation as competitive advantage
Building and
• Safety value: reduce accidents due to human errors Deployment:
acquisition, trials on
• Efficiency ? selected roads, backup
driver training, pilots,
implementation
• How to ensure the responsible, compliant, effective and efficient
operations?
• Policies and procedures to guide decisions and actions in capability Operation and
Monitoring:
planning, building and operating operational support,
evaluation, compliance

• Controls to track performance and ensure compliance - pinpoint

problems, assigning responsibilities to individuals and groups.
Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 9
Ethics & Regulation Develop, implement, and use AI solutions that are both
ethically responsible and also legally defensible.

• Establish a • Have a clear picture of • Define a practical

comprehensive and relevant legal approach operational to
solid ethical framework obligations that directly ethical AI guidelines.
for the organisation in regulate AI applications. • Develop a process to
the space of AI.
ensure compliance with
the legal obligations.
• Define clear guidelines • Review and adapt
to the development and processes to manage
use of AI applications. responsible AI.

Contextualise Define a Relevant Operationalise

Ethically and Legally
AI Ethics Legal Framework RAI

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 10
AI Performance

• AI must be resilient and • Instil trust in AI systems • Tune AI systems to

reliable: detect and mitigate bias, not using
• Be able to explain to
correct faults and data resulting in bias,
various stakeholders
inaccurate decisions. check outcomes,
why a particular AI
• Built-in security model reached a transparency
covering all AI systems, particular decision. • Adhere to corporate
data, and networks. code of ethics and
• regulators, data
• Safe for the people scientists, business following anti-
whose lives they affect sponsors, and end discrimination
consumers regulations.

Robustness and Interpretability &

Security Explainability Bias and Fairness

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 11
 Explainable AI
(XAI)

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 12
Deep Learning example

• Patient data and clinical data are specified as

inputs x1, x2, x3... (age, gender, med. compliance,
#procedures, LoS, type of visit, #emergency_visits,
#prior_admissions, etc)
• Readmission risk score can be specified as the label
output (y0)
• Neural network models can be trained, tested and
selected for use in predict readmission for new
visits/discharges

For example

https://static.javatpoint.com/tutorial/ai/images/deep-learning-works.png

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 13
Explainable Artificial Intelligence (XAI)

Dam et al. (2018)

Lipton (2017)

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 14
Explainable Artificial Intelligence (XAI)

Interpretable Explainable Transparent Justifiable Contestable

some sense of users can understand some level of there is an users have the
understanding how the why or how a accessibility to the data understanding of the information they need
technology works conclusion was reached or algorithm case in support of a to argue against a
particular outcome decision or
classification

Adapted from The Royal Society (2019)

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 15
Generic stakeholders in Artificial Intelligence

Regulatory Authorities Data Scientists and Business Managers End Users End Consumers
systems developers
Provide standards, guidelines Influence use of data and Sponsor AI systems Use AI directly Affected / benefit from AI-
and Monitor compliance algorithms enabled decision making

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 16
Stakeholders in Artificial Intelligence

https://opendatascience.com/ibm-research-launches-explainable-ai-toolkit/

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 17
Who needs this XAI type?

Hepenstal and McNeish, 2020

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 18
 The Royal Society (2019)
https://royalsociety.org/-/media/policy/projects/explainable-ai/AI-and-interpretability-policy-briefing.pdf

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 19
Criminal Justice – case study

Criminal justice risk assessment tools

Concerns in recent years:… fairness and the ability to contest a
Inputs: individual’s characteristics (demographics,
record of offences, and so on) decision.

Outputs: predictions about the likely future In some jurisdictions, there already exists legislation against the use of
behaviour of repeat offenders protected characteristics – such as race or gender – when making
decisions about an individual’s likelihood of reoffending. These features
can be excluded from analysis in an AI- enabled system.
..such tools offer the hope of a fairer system, in
which human bias or socially-influenced however …their association with other features can ‘bake in’ unfairness
perceptions about who is a ‘risk’ are less likely to in the system; for example, excluding information about ethnicity but
influence how an individual is treated by the justice including postcode data that might correlate with districts with high
system. populations from minority communities.

...the possibility of increasing the accuracy and Without some form of transparency, it can be difficult to assess how
consistency of these predictive systems. such biases might influence an individual’s risk score.

Extracted from The Royal Society (2019)

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 20
Criminal Justice – case study

Criminal justice risk assessment tools

Inputs: individual’s characteristics (demographics, Without some form of transparency, it can be difficult to assess how
record of offences, and so on) such biases might influence an individual’s risk score.
Outputs: predictions about the likely future
behaviour of repeat offenders Some challenges:
• lack of transparency …due to IP protections and trade secrets
• the risk of revealing sensitive personal information

..such tools offer the hope of a fairer system, in Proposals to address these concerns :
which human bias or socially-influenced • provision of additional information to judges and those working in
perceptions about who is a ‘risk’ are less likely to the justice system to help them interpret the results of a system,
influence how an individual is treated by the justice
system. and additional training for those individuals;
• the use of confidence estimates to help users interpret the results
...the possibility of increasing the accuracy and of a system;
consistency of these predictive systems. • systems to evaluate, monitor, and audit algorithmic tools.

Extracted from The Royal Society (2019)

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 21
Health – case study

Medical imaging tool: AI systems will need to work well in clinical practice:
• clinicians and patients may both want to understand the reasoning
behind a decision.
Inputs: scans of tissue samples
• ...especially when the treatments that follow can have life-altering
Outputs: diagnosis - the likely presence or absence effects
of disease
Example of Explainable AI
DeepMind and Moorfield’s Eye Hospital’s XAI analysed over 14,000
retinal scans.

Such tools inform clinicians’ decisions about XAI was built in to help clinicians understand why it had made a
treatment pathways. Challenge is complexity diagnosis recommendation by making it decomposable:
requiring expertise to analyse and interpret. • 1st ANN analyses a scan to detect and create a map of features in
the image that are illustrative of the presence (or absence) of
disease.
• 2nd ANN analyses this map, using the features to present clinicians
with a diagnosis with a percentage to illustrate confidence of results

Extracted from The Royal Society (2019)

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 22
A2: Q&A

A2 Project

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 23
Upcoming next week – A2 Consultation

A2 Project
Consultation

AIR and RAI

Consolidation

Deakin University CRICOS Provider Code: 00113B - A/Prof Lemai Nguyen MIS716 AI for Business Slide 24