NextGen Risk Management

How Do Machines Make Decisions?

Internal Audit, Risk, Business & Technology Consulting

Introduction

Effective risk identification and monitoring are integral to an organization’s success and
improving strategic decision-making. Accurate and timely risk identification and assessment
help drive efficiencies and improve customer experiences with business processes.

Consistent with its agile risk management philosophy, Protiviti presents its perspective
on establishing and sustaining leading practices for identifying, assessing, mitigating and
monitoring risks stemming from artificial intelligence (AI).

Value of Agile Risk Management

• Customer Centricity
Customer
Satisfaction • Consistent Experiences

• Agility

Risk • Optimized Performance

Management
• Focus on Growth
Aligned Operational
Organization Excellence
• Risk-Enabled Decisions

Protiviti’s Agile Risk Management philosophy enables organizations to focus on growth, improve efficiency
and become more effective at managing risks while providing greater value to business partners.

Source: Protiviti Insights — Agile Risk Management: "As costs continue to increase, it is clear that the overly manual, reactive and siloed lines of defense
status quo is unsustainable and cannot continue. We believe risk capabilities must be agile, flexible and nimble in order to be effective and efficient in
responding to the changing environment. A better model is technology-enabled, proactive, aligned across all three lines of defense and embedded into
business processes. This is the solution we refer to as Agile Risk Management.”

protiviti.com Next Generation Risk Management · 1

AI and Risk Management

Many organizations are quickly adopting AI based
on the benefits it can create. AI technologies have the
potential to advance established industries by improving
the efficiency and accuracy of company operations and
customer experiences. Additionally, AI is opening the
door to entirely new operating models, ushering in a new
set of competitive dynamics that rewards organizations
focused on interpreting and extracting internal and
external data quickly and accurately. 1
Machine learning, a type of AI, utilizes the fields of
knowledge discovery and data mining. Machine learning
algorithms study and react to data automatically,
without human assistance or intervention, enabling
systems to learn from experience and improve.
However, using machine learning and AI increases
complexity and creates new, more dynamic risks that
may lead to unintended consequences.
To mitigate the new and changing risk environment,
an organization needs to have a properly established
risk management foundation. Organizations can
leverage existing risk management frameworks to
create a framework that can identify and oversee the
wide range of risks associated with AI. For instance,
risk frameworks utilized to assess new products and
services, as well as activities, can be leveraged, as AI
is developed, implemented and changed. Another
useful framework is a model risk management (MRM)
framework that is based on identifying, measuring
and monitoring all risks related to a model —
generally a component of AI in the form of a machine
learning algorithm.
MRM practices mitigate the risks of traditional
econometric model lifecycles, however, often they
fail to capture the risks presented by AI. While these
frameworks can be leveraged, organizations may not
be currently equipped and resourced to handle all risks
and ongoing monitoring needed in an AI environment.
To account fully for risks posed by AI, organizations’
existing frameworks and risk practices can be tailored
with some well-targeted enhancements within the AI
lifecycle, as discussed in detail below.
As use of AI continues to expand exponentially, risk
and compliance functions will be challenged to rethink
resourcing, traditional oversight monitoring techniques,
and how to leverage existing frameworks to ease
implementation and fully manage risks.

AI technologies have the potential to advance established industries by improving the efficiency and accuracy
of company operations and customer experiences. Additionally, AI is opening the door to entirely new operating
models, ushering in a new set of competitive dynamics that rewards organizations focused on the scale and
sophistication of data much more than the scale or complexity of capital.

“The New Physics of Financial Services: How Artificial Intelligence Is Transforming the Financial Ecosystem,” World Economic Forum, Aug. 15, 2018: www.weforum.org/
1

reports/the-new-physics-of-financial-services-how-artificial-intelligence-is-transforming-the-financial-ecosystem.

2 · Protiviti
AI in the Marketplace

The financial services industry continues to invest
heavily in artificial intelligence systems, leading other
industries such as manufacturing, healthcare and
professional services. Last year, research firm IDC said
it expected the banking industry to spend more than $5
billion on artificial intelligence systems in 2019. Overall,
IDC projects spending on AI systems will reach $97.9
billion in 2023, more than two and one half times the
$37.5 billion that will be spent in 2019. 2
Financial institutions are incorporating AI into asset
management, fraud detection, credit risk management
and regulatory compliance, to name a few use cases.
Specifically, these organizations are turning to machine
learning models as an alternative to traditional models
to gain faster, more accurate, and insightful predictions
and classifications in their risk management and financial
management business decisions. Several types of AI
components and the effect they have on organizations are
provided below.

AI Use in the Marketplace

Component Operating Efficiencies

Machine Learning Models Organizations can use AI as a modeling technique through machine
learning to improve decision-making in these select areas:

• Underwriting/credit decisioning
• Personalized marketing
• Asset management
• Compliance monitoring
• Credit risk management
• Customer segmentation
• Fraud detection
• Loss forecasting

Virtual Agents Virtual financial assistants or chatbots can guide consumers through
day-to-day financial tasks, providing personalized and proactive
assistance to help them stay on top of their personal finances.

Natural Language NLP enhances organizations’ ability to analyze countless numbers of

Processing (NLP) documents, including contracts, emails and forms, enabling them to
better quantify and examine available data that would otherwise be
difficult and inefficient to extract from unstructured source material.

Image Analysis AI-powered image analysis can be used by organizations to classify

images and trigger real-time actions based on image data capture,
enhancing the customer experience process. For example, insurers are
using image analysis to capture and analyze images of homes damaged
after a natural disaster, increasing the efficiency of claims processing.

2
IDC Worldwide Artificial Intelligence Spending Guide: www.idc.com/getdoc.jsp?containerId=prUS45481219

protiviti.com Next Generation Risk Management · 3

Incorporating and monitoring AI the correct way is
important. There have been several instances where
major organizations have rushed to deploy AI, only
to learn of the unmitigated risks and unintended
consequences of their application. In 2018, a major
consumer brand discovered that the AI used in its
hiring process discriminated against female job
applicants. The software was designed to align a
candidate’s history with that of employees who had
proven successful at the company over the previous
10 years.3 The design of the algorithm did not
intend to discriminate but the data set on which the
model relied caused unintended consequences and
bias. The following table shows common risks that
organizations are encountering through the use of AI:

Key Risks Posed by AI

Common Risks of AI

Regulatory and
Strategic Risk Operational Risk Technology Risk Financial Risk
Compliance Risk

• Reputational Risk • Legal Risk • Business Disruption • Software/Application • Credit Risk

Failure
• Customer • Consumer Protection • System Failures • Liquidity Risk
Experience
• Know Your Customer • Process Failures • Information & Cyber
• Market Risk
Risk
• Stakeholder Risk (KYC)
• Internal Control • Underwriting Risk
• Resource • Consumer Privacy Environment • Identity & Access
Allocation
Management • Financial Reporting
• Disparate Impact • Third-Party Risk/
• Availability &
Risk
• Culture
• Unfair, Deceptive,
Vendor Management
Accessibility
• Obsolete or Abusive Acts or • Change Management
• Black-Box Issues
Workforce Practices
• Operational Errors
• Data Management
• Talent • Fair Credit & Lending
Management
• Sales Practices/ • Data Security
• Brand Awareness Incentive Comp

Forbes Insights: www.forbes.com/sites/insights-intelai/2019/03/27/ai-regulation-its-time-for-training-wheels/#5981d0cc2f26

3

4 · Protiviti
Although AI is innovative and technically complex, it has
foundational components of a core model that quantifies
theories, techniques and assumptions from processed
input data. However, the differences with AI are the
exponential increase of model complexity due to intricate
algorithms, vast unstructured data sets and the potential
for immense decision trees. AI — specifically, machine
learning — removes the element of human subject-
matter expertise from the decision process, which can
result in unwanted risk exposure.
As the use of machine learning models continues to
expand across the financial services industry, regulators
are increasing their attention on model risk. The
following three root causes can result in model risk:
• A model has fundamental errors that cause it to
produce inaccurate or biased outputs when viewed
against the design objective and intended business use.
• A model is implemented or used inappropriately,
or when its limitations or assumptions are not
fully understood.
What Are We Learning about Artificial Intelligence in Financial Services?: www.federalreserve.gov/newsevents/speech/brainard20181113a.htm
4
Validation of Machine Learning Models: Challenges and Alternatives: “www.protiviti.com/US-en/insights/validation-machine-learning-models-challenges-and-alternatives”
5
protiviti.com
• A model is misused because of a misunderstanding
of its purpose and limitations.
To avoid these challenges, organizations should consider
these fundamental questions:
• Do you know how the machine learning model
was built?
• Do you know its purpose?
• Do you know how to use the results and how
success is defined?
The Federal Reserve Board (FRB) has reinforced that SR
11-7/ OCC 2011-124 (Guidance on Model Risk Management)
remains the applicable regulatory guidance on the use
of AI. There have been no indications by the FRB of any
new standards or requirements that will come into place.
Although SR 11-7/ OCC 2011-12 provides a foundation for
establishing risk management frameworks for mitigating
risks posed by AI systems, guidance and expectations
have not been expanded and formalized to address the
dynamic changes, unintended results, and bias risks5
posed by AI.
Next Generation Risk Management · 5
Organizations can proactively mitigate these unique
AI risks by establishing cross-functional frameworks,
based on a clearly defined scope of each AI solution and
interdependencies with existing risks in its operating
environment. Consider the use of a chatbot as an
example. An organization will need to consider legal,
compliance, reputational and operational risks if any
issues (discrimination, bias, privacy, etc.) arise from
the use of a chatbot.
AI Lifecycle and Effective Challenge
Perform model redesign and
recalibration
Review process for AI
model findings
Analyze and review
AI modifications
Perform post-implementation
model validation
Review performance threshold
exception reports
Risk & Compliance Monitoring
6
NYDFS Apple Card Investigation: www.bankingdive.com/news/apple-card-investigation-alleged-gender-discrimination/567050/
protiviti.com
Recently, the New York Department of Financial
Services launched an investigation into gender
discrimination in financial institutions’ consumer
algorithms that are used to determine credit limits.6
Needless to say, organizations using AI for decisions
are facing scrutiny across the board as it relates to the
risk taxonomy. Given these challenges, organizations
should enhance their current risk management
framework by establishing a cross-functional risk
governance process to ensure AI risks are understood,
assessed, and mitigated throughout the AI lifecycle.
Request the AI model
1
Conduct preliminary analytics and design
12 2
Develop the AI model
D
es
t
es
3
ign
11
lement and T
and M ate Ris
Effective
Challenge
itig
10 4 Validate the AI model
before implementation
Imp
9 5 Finalize the AI model
8 6
Implement the AI
7
model into production
AI model owners monitor
performance
Internal Audit Reviews
Next Generation Risk Management · 6
Insight into the lifecycle will help organizations navigate various considerations, including risk and compliance,
governance and reporting, data management, technology, and workforce and training implications. Additionally, an
environment of effective challenge, where decision-making processes promote a range of views, fosters independent
testing and validation of current practices and AI solutions prior to implementation and production, and an integrated
environment of open and constructive engagement. Organizations can take the following actions now to enhance risk
mitigation during the AI lifecycle:

1  Design and Mitigate

AI Governance Build-Out

• Adapt and extend existing model governance to fit AI • Configure a risk-based methodology consisting of
tools, specifically the use and maintenance of models, severity tiers, which will incorporate the necessary
validation of models, and the adequate disclosure of requirements to implement AI successfully.
model assumptions and limitations.
• Formalize a well-defined project oversight and change
• Review and update the model risk policy regulating management framework around AI systems.
the definition of model risk, scope of MRM, roles
• Improve data quality programs to profile input data
and responsibilities, model approval and change
and strengthen data governance (i.e., embed data
process and management of model weaknesses, to
requirements and a rigorous data monitoring process).
encompass the new risks that AI presents.
• Build a data warehouse for all performance monitoring
• Develop an AI policy consisting of requirements
and testing data. This will allow an AI tool to easily
around use, development, and ongoing monitoring,
input and manage the data repository once the
which include roles and responsibilities for business
structure is built.
leaders, independent risk and compliance managers,
and technology and operations functions. • Configure application resiliency controls, detailed
business-continuity planning and disaster recovery.
• Determine the interoperability requirements based on
the organization’s risk appetite as part of the AI policy. • Track and aggregate monitoring in centralized
warehouses and align to issue and change
• Develop a methodology around bias to ensure
management programs.
fairness and address algorithmic bias, as well as bias
against humans.

7 · Protiviti
AI Tool Design
• Define the purpose and scope of the AI solution
clearly, including its methodology, decision criteria,
and data requirements.
• Hold meetings with key stakeholders to understand
the AI tool requirements, desired output and use cases.
• Before developing an AI tool, map its process
workflow, including data inputs, variables, and
monitoring triggers to gain a full understanding of the
foundation of the tool.
• Complete documentation of the AI tools underlying
model’s purpose, design, assumptions, parameteriza-
tion, testing, limitations, and user instruction.
• Identify scale and potential inherent risks that may be
triggered with the use of an AI solution.
• Examine the amount of change that a business will
be required to undergo as it relates to building and
running the AI tool in production.
• Embed, understand and analyze rules and regulatory
requirements in the algorithm design and monitoring.
2  Implement
• Ensure the approved project plan serves as the
baseline or source of record, and acts as a “contract”
of the work to be performed to successfully
implement the AI tool.
• Hold meetings with key stakeholders to introduce
the AI and designate model owners and SMEs to
monitor performance.
• Configure a cross-functional team consisting of
data scientists, AI experts, model risk experts,
data officers, regulatory experts, and any key
protiviti.com
• Define hyperparameters, including a standard set of
analysis to be run on input data and output results.
• Perform quality control during pre-implementation
rollout.
• Obtain appropriate approvals and signoffs for
development and use of the AI tool.
• Build mechanisms within the AI tool to ensure
accountability and adequate access to redress.
Algorithms, data and design processes should all
be auditable.
• Configure consistent and recurring testing in a live
environment.
• Conduct preliminary analytics on the outputs
generated by the tool to understand its limitations
and determine optimal parameters when building
out the tool.
• Validate the parameters chosen through human
subject-matter experts (SMEs) and industry
benchmarks.
stakeholders to help mitigate risks associated with
the implementation of the AI tool.
• Establish and monitor controls and human override
in the design of the algorithm to control inputs,
processing and outcomes during implementation.
• Conduct proof-of-concept testing and/or controlled
case studies before going into live production.
• Develop an implementation plan for moving the
AI solution into production and assist with the
implementation phase.
Next Generation Risk Management · 8
• Develop and formalize communication protocols to
internal and external stakeholders (e.g., consumers,
investors, regulators) of the use of the newly
implemented AI tool.
• Perform a production readiness analysis to ensure the
AI solution can be implemented successfully.
3  Testing and Effective Challenge
• Perform rigorous and continuous testing of
underlying/input data.
• Perform scheduled backups and parallel testing of
underlying/input data.
• Conduct periodic testing of the controls in place to
guardrail underlying/input data.
• Perform post-implementation AI validation testing
and exceptions testing and conduct a risk assessment.
• Review AI model findings and hold meetings with key
stakeholders and SMEs to discuss key takeaways.
• Review performance threshold exception reports to
identify areas of improvement for the model.
• Formalize review of key risks inherent in AI and its
operational component (e.g., economic variables,
qualitative factors).
• Perform a quality assurance review of surrounding
business objectives, stated benefits and process flow.
• Review choice of architecture, hyper-parameters,
optimizers, regularization and activation functions.
• Conduct an independent assessment as it relates
to operating within parameters outlined in the
approval documentation.
• Modify parameters dynamically to reflect emerging
patterns in the input data, as this will replace the
traditional approach of periodic manual review and
model refresh.
protiviti.com
• Perform validation testing of the AI tool prior to
implementation and make final updates to mitigate
any material weaknesses of the tool.
• Provide insight regarding risk and compliance
considerations that align to the use of AI.
• Conduct an independent audit to ensure the design
and effectiveness of controls relied upon to mitigate
the model’s risks.
• Perform an independent assessment of the process for
establishing and monitoring limits on model use.
• Conduct a bias/variance analysis.
• Develop a challenger model using alternative
algorithms to benchmark output performance.
• Perform a post-implementation analysis to determine
if the change management process or methodologies
need to be modified.
• If needed, redesign and recalibrate the AI model
based on the findings, discussions, and risk and
compliance considerations.
• Incorporate appropriate human intervention
throughout each component of the AI lifecycle.
• Develop an AI feedback loop consisting of existing
complaints and customer feedback to allow an
organization to understand and quickly resolve AI
issues and/or defects.
Next Generation Risk Management · 9
AI Risk Management Framework

Numerous organizations are intensely focused on gaining
a competitive advantage through AI implementation. To
succeed, organizations need to commit to monitoring and
understanding risks posed by AI.
As AI becomes more prevalent, it is crucial for
organizations to move into an agile risk target state to
manage AI risks. An organization can align its MRM
infrastructure with the enhanced procedures and
controls, while incorporating new AI activity governance,
agile implementation and effective challenge of AI
tools. Establishing an AI risk framework will benefit
an organization’s ability and speed to innovate. This
can be applied to all three lines of defense and updated
regularly to reflect evolving best practices and regulatory
expectations. The updated framework can leverage
existing governance and risk management activities
while catering to AI.

AI Risk Management Framework

• Analysis of Findings • Policy & Procedures

• Findings Prioritization • Lifecycle Standards
7 •
•
Roadmap for Implementation
Redesign/Recalibration for
1 •
•
Approval & Accountability
Risk Oversight
Continuous Improvement • Change Management

• Output Analysis
Post-Mortem Governance • Al Identification

• Interpretability
Review
• Al Inventory

• • Applicability
6 •
Bias Testing
Operational Issues
AI Risk
2 •
•
Risk Assessments
Risk Ratings
• Review of Performance Indicators Independent
Validation
Management
Inventory &
Risk Assessment • Model Impact Assessment
• Review of Recommendations
(Risk Scoring)
Framework

• Testing Program Ongoing

Performance
Data Aggregation
& Quality • Data Architecture
• •
•
Effective Challenge
Stress Testing
Monitoring
Integrated 3 •
Data Infrastructure
Data Privacy
• Real-Time Monitoring and Bias Development &
Implementation • Feature Engineering
Output Reporting
•
5 •
Dynamic Model Calibration
Results & Output Based Testing 4
• Proactive Trend, Concentration &
Correlation Identification • Data Quality Assessment
• Benchmarking • Testing & Analysis
• Continuous Automated Exception • Control Framework
Identification & Reporting • Secure Data Model
• Training
• Pre-Implementation Validation
• Hyperparameters
• Production Readiness
• Model Input Change Management

10 · Protiviti
With an agile AI risk framework, organizations should, at a minimum, implement the following activities and concepts
per the framework components:
1  Governance
• A formalized governance structure will establish
accountability around the execution of the AI lifecycle.
It will also assign appropriate resources and processes
required to assess the design and performance of the
AI tool.
• Organizations will be required to ensure resources
possess the appropriate skill sets needed to challenge,
control, and monitor the use of AI. However, due to the
complexity of AI, the respective skill set to govern AI
effectively will be tailored for the sustainability and
for each business use of the AI tool.
- For example, a line-of-business SME will be
needed to verify if the expected AI outputs are
achieved, while a technology SME is needed to
2  Inventory & Risk Assessment
• Organizations will immediately need to revisit
their tools inventory to ensure AI models are
included. A robust model inventory provides
management with a comprehensive overview of all
models in use, including model owners, restrictions
on use, and the validation status. Lack of a robust
method to update the model inventory on a regular
basis can result in undocumented model changes,
inefficient processes to risk rate models, and
ineffective performance monitoring.
protiviti.com
verify if the AI was efficiently integrated into
an organization’s technological infrastructure
without falling into algorithmic loops that
overload the system.
• With the enhancement of the governance structure,
organizations will need to incorporate the following:
- A formalized, documented, clear, and
comprehensive definition of AI.
- Defined roles and responsibilities.
- A formalized and socialized project
governance charter.
- A formalized and responsive change
management process.
• The organization’s model risk assessment process,
as required under regulatory guidance, will need
to be formally adapted to incorporate AI. The risk
assessment process will need to assess model impact
risk, covering both the assumptions that are drawn
from models and the impact of decisions based
upon model output. Conducting a risk assessment
allows an institution to understand inherent risks
of the business, products and services, as well as
the effectiveness of the controls in place. A periodic
risk assessment will support appropriate scheduling
of monitoring to ensure resources are allocated and
risk is mitigated.
Next Generation Risk Management · 11
3  Data Aggregation & Quality
• Organizations will need an effective and transparent
process to improve underlying or input data
throughout the model’s tenure. A formalized and
documented model input change management
process and communication plan is critical to the
aggregation and quality of underlying or input data
used in the AI tool. The key stakeholders (model
owner, model user, model approver, and
4  Integrated Development & Implementation
• The successful development and implementation of
AI solutions within an enterprise depends largely
on the design and effectiveness of the control and
testing process. An enhanced control framework and
continuous testing can help reduce inherent risks to a
residual risk level that aligns with the organization’s
risk appetite and framework. Currently, organizations
tend to test new initiatives within a sandbox
environment; however, given the complexity and
development of AI, they should consider configuring
consistent and recurring testing outside a sandbox.
Developing a control framework and testing process
would allow organizations to identify gaps and
potential options for improvement quickly. The
control process should be determined and aligned
by an established and enhanced risk assessment
framework. The risk assessment process is critical, as
it helps to determine the controls needed to mitigate
the inherent risks.
12 · Protiviti
independent reviewer) will be required to maintain
and/or understand the following components:
- Data quality and data set integration.
- Data architecture and data infrastructure.
- Understand > review > assess > remediate >
algorithms.
- Transparency of algorithms.
- Effective controls in place to guardrail
underlying/input data.
• Organizations should consider the key risks generated
from the use of AI. For example, data bias will require
organizations to produce impartial decisions by
examining the choice of data. As bias in AI can trigger
costly errors, organizations will need to focus on the
front-end of the AI lifecycle, the development of the AI
tool. One way to identify data bias is by benchmarking
with other models or the opinion of SMEs. Appropriate
data de-biasing techniques should be used to remove
bias from development data. In addition to traditional
methods such as downscaling and quantile mapping,
randomization and sample weighting should also
be incorporated to correct data bias. The statistical
soundness of selecting unbiased development and
holdout data should be given extra emphasis for
machine learning models.
5  Ongoing Performance Monitoring
• Performance monitoring is essential to mitigating
risks connected to AI tools. Effective monitoring
will help an organization draw clear conclusions to
support business decisions. An effective performance
monitoring function comes from a highly automated
monitoring and testing program, using a common
methodology and real-time reporting. Organizations
can enhance the rigor of the performance monitoring
function by using the techniques below:
- Real-time monitoring and bias output reporting.
- Results and output-based testing.
- Proactive trend, concentration and
correlation identification.
- Assurance of appropriate and compliant
recommendations.
- Continuous automated exception identification,
alert system and reporting.
- Proper skill set.
- Repurposing workforce.
- Reskilling workforce.
• Additionally, it will be critical for organizations to
- Multidisciplinary team structure with formal
project management.
• Effective challenge requires the cooperation and
alignment of all three lines of defense, as each plays
a specific role. The first line of defense, specifically
model developers and owners, works to understand
and monitor the risks from the use of an AI tool. The
second line, the model validators, independently
protiviti.com
establishes key protocols for risk and compliance
decisions while working with model developers and
owners. Lastly, the third line of defense, specifically
audit, conducts its own tests to ensure that the
residual model risk of the AI tool does not surpass the
risk appetite established. The scope of activities by
the third line of defense will stay similar in nature
in comparison to the traditional MRM framework.
However, the third line of defense will be required to
expand its skill set to understand how AI algorithms
work and their intended use, as well as understand
the risk they pose to technology infrastructure and
operations. To have the most impact, an effective
challenge must include the following:
- Two-way communication on strategic business and
risk decisions as it relates to the use of the AI tool.
- Transparency and direction to business and risk
leadership before issues arise from the use of
the AI tool.
- Full use of the AI tool according to the
established risk appetite.
maintain human subject-matter oversight rather
than strictly relying on software solutions to
render analysis, as software has the potential to
fail to understand the impacts of the results. Lastly,
organizations should review and update policy,
procedures and processes periodically to encompass
the changes that AI brings, which, in turn, will help an
organization effectively evaluate an AI tool.
Next Generation Risk Management · 13
6  Independent Validation
• As with any model, periodic independent
validations will continue to be a focal point of
7
AI monitoring. To assess the innovations of AI,
model validators will need to understand the
challenges, such as a model’s fitness for use, and
develop customized methods for validating AI
tools. The validation will still be required to assess
models broadly from four perspectives: conceptual
soundness, process verification, ongoing
monitoring and outcomes analysis.
7  Postmortem Review
• An organization will need to plan strategically
and execute effectively around the performance
monitoring results, as postmortem reviews will
be crucial to refining and improving the models.
Organizations will need to thoroughly examine the
analysis and explanation of the AI output, bias and
7
Validation of Machine Learning Models: Challenges and Alternatives: www.protiviti.com/sites/default/files/united_states/insights/validating-machine-learning-models-
whitepaper-protiviti.pdf
protiviti.com
• SR 11-7 and OCC 2011-12 require that model
documentation be comprehensive and detailed
enough so that a knowledgeable third party can
recreate the model without having access to the
model development code. The complexity of AI and
the model development process are likely to make
documentation of AI tools much more challenging
than traditional model documentation. It is
recommended that organizations standardize their
model development and validation procedures for AI
and provide a model documentation template that is
consistent with regulatory expectations and its model
risk management policies and standards.
interpretability analysis, and review performance
threshold exceptions and controls in place. Based on
the examination and reviews, organizations will need
to constantly redesign and recalibrate the AI tool for
continuous improvement.
Next Generation Risk Management · 14
Conclusion
With the continued investment in AI, the use of AI in
business processes and practices is only growing larger
in scope and deeper in granularity. To stay ahead and
provide effective and efficient monitoring of risk,
organizations will not only utilize AI as their most
comprehensive and valued tool but will need agile risk
and compliance management. Competitive advantages
will come not only from how organizations use AI but
also from how they are able to avoid mistakes, ensure
smooth customer experiences, prevent violations of
law and explain what AI is intended to do to customers
and regulators.
15 · Protiviti
An AI tool will never be fully clear of risk, but an
efficient and effective AI risk management framework
will keep risk manageable and enable organizations to
respond to fluctuations in the outputs and decisions
generated by AI. The key for all organizations using AI
currently is to build and maintain AI in a responsible
and transparent way, which, in turn, will help reduce
operational cost and, more important, maintain the
confidence of customers.
ABOUT PROTIVITI

Protiviti is a global consulting firm that delivers deep expertise, objective insights, a tailored approach and unparalleled collaboration to help leaders confidently
face the future. Through its network of more than 85 offices in over 25 countries, Protiviti and its independent and locally owned Member Firms provide clients
with consulting solutions in finance, technology, operations, data, analytics, governance, risk and internal audit.

Named to the 2020 Fortune 100 Best Companies to Work For® list, Protiviti has served more than 60% of Fortune 1000 ® and 35% of Fortune Global 500 ®
companies. The firm also works with smaller, growing companies, including those looking to go public, as well as with government agencies. Protiviti is a wholly
owned subsidiary of Robert Half (NYSE: RHI). Founded in 1948, Robert Half is a member of the S&P 500 index.

HOW PROTIVITI CAN HELP

Protiviti has a record of success helping clients develop strong risk management practices with the responsiveness required for an ever-changing business
environment. We work with over 75% of the world’s largest financial institutions, which benefit from our collaborative team approach to resolving today’s risk
management challenges. Our professional consultants have varied industry and regulatory backgrounds that enable our unified financial services practice,
with the seamless integration of risk and compliance, technology, data and analytics solutions, to develop customized agile risk management approaches to
meet tomorrow’s challenges today.

Business, risk, compliance and internal audit groups need to work within an integrated framework with clear accountabilities that will lead to an aligned
organization for making sound decisions. We address risk and operational excellence as two sides of the same coin, leading to agility and optimal performance.
We understand how customer satisfaction, and in turn growth, have become elusive. While risk management is intended to drive growth, it too often becomes
an inhibitor. Our expertise positions you at the forefront of effective risk management with a unique approach to reap both immediate and long-term benefits.

CONTACTS

Matthew Moore Michael Brauneis Madhumita Bhattacharyya

Managing Director and Managing Director and Managing Director
Global Risk and Compliance Leader Americas Financial Services Leader +1 469-374-2564
+1.704.972.9615 +1.312.476.6327 madhumita.bhattacharyya@protiviti.com
matthew.moore@protiviti.com michael.brauneis@protiviti.com
Shaheen Dil
Suresh Baral Matthew Perconte Managing Director
Managing Director Managing Director +1.212.603.8378
+1.212.471.9674 +1.212.479.0692 shaheen.dil@protiviti.com 
suresh.baral@protiviti.com matthew.perconte@protiviti.com

Lucas Lau
Director
+1.212.603.8398
lucas.lau@protiviti.com

protiviti.com Next Generation Risk Management · 16

 © 2018 Protiviti Inc. An Equal Opportunity Employer M/F/Disability/Veterans. PRO-0918
THE AMERICAS UNITED STATES
Alexandria
Houston
Kansas City
Sacramento
Salt Lake City
ARGENTINA*
Buenos Aires
COLOMBIA*
Bogota
Atlanta Los Angeles San Francisco
Baltimore Milwaukee San Jose BRAZIL* MEXICO*

Boston Minneapolis Seattle Rio de Janeiro Mexico City

Sao Paulo
Charlotte New York Stamford
PERU*
Chicago Orlando St. Louis CANADA Lima
Cincinnati Philadelphia Tampa Kitchener-Waterloo
Cleveland Phoenix Washington, D.C. Toronto VENEZUELA*
Dallas Pittsburgh Winchester Caracas
Denver Portland Woodbridge CHILE*
Fort Lauderdale Richmond Santiago

EUROPE, FRANCE
Paris
NETHERLANDS
Amsterdam
BAHRAIN*
Manama
SAUDI ARABIA*
Riyadh
SOUTH AFRICA *
Durban

MIDDLE EAST GERMANY SWITZERLAND KUWAIT* UNITED ARAB

Johannesburg

& AFRICA Frankfurt

Munich
Zurich Kuwait City EMIRATES*
Abu Dhabi
UNITED KINGDOM OMAN* Dubai
ITALY Birmingham Muscat
Milan Bristol EGYPT*
Rome Leeds QATAR* Cairo
Turin London Doha
Manchester
Milton Keynes
Swindon

ASIA-PACIFIC AUSTRALIA
Brisbane
CHINA
Beijing
INDIA*
Bengaluru
JAPAN
Osaka
Canberra Hong Kong Hyderabad Tokyo
Melbourne Shanghai Kolkata
Sydney Shenzhen Mumbai SINGAPORE

New Delhi Singapore

*MEMBER FIRM

© 2020 Protiviti Inc. An Equal Opportunity Employer M/F/Disability/Veterans.

Protiviti is not licensed or registered as a public accounting firm and does not issue
opinions on financial statements or offer attestation services. PRO-0320-103142