GFMI Model Risk Conference, September 2020

Robert Serena

Model Risk for insurance companies

About the author
Robert Serena, FSA, CPCU, CFA, FRM, CRISC

Mr. Serena is a Risk Management and Actuarial executive with a very unique blend of financial services functional
experience across insurance, reinsurance, commodity trading, and commercial banking - numerous technical and
leadership roles in the First Line-of-Defense (Actuarial, Investment Management, and Capital Markets & Trading) and
Second Line-of-Defense (Risk Management and Compliance).

He holds a BS in Electrical Engineering from Rice University, an MS in Operations Research from the University of
New Haven, and several professional certifications – Fellow in the Society of Actuaries (FSA), Chartered Financial
Analyst (CFA), Financial Risk Manager (FRM), Chartered Property Casualty Underwriter (CPCU), and Certified in Risk
and Information System Control (CRISC).

He currently lives in the Charlotte, NC area with his wife and two children.
 Executive Summary
• The purpose of this presentation is to provide a broad overview of current practices for Model Risk Management (MRM) in the
insurance industry – both Life, Annuity and Health (LAH) and Property & Casualty (P&C).

• I look at a variety of factors that are impacting the practice of MRM in insurance:
• Alternative definitions of models and model risk
• Where MRM slots into the Enterprise Risk Management (ERM) hierarchy
• Sources of Model Risk and specific use cases
• Business applications for modeling
• Regulatory Drivers
• Roles that actuaries can play in managing model risk
• Skills required to be an effective model risk practitioner
• Hypothetical use cases for advanced analytics

• There are a wide range of drivers that are increasing model risk for insurers:
• More competitive marketplace with increased capital markets volatility, shrinking profit margins, and greater customer demands for
product customization; and
• Increasing reliance of insurers on a wider variety of internal and external data sources, and more complex analytics ecosystems to
manage their respective businesses profitably; and
• Heightened demands on models with regards to speed, granularity, and capacity; and
• Increasing regulatory requirements and expectations related to model development and model governance.
Enterprise Risk Hierarchy - Life, Annuity & Health Insurer

Enterprise Risks

Market Risk Credit Risk Insurable Risk Operational Strategic Compliance

Interest Rate Spread Mortality Systems Process People Competitive Legal breach

Information Inadequate Fraud &

Equity Price Default Morbidity Sourcing Talent Regulatory breach
Security training Misconduct

Implementation Inadequate Contractual

Foreign Exchange Migration Longevity Product trends
Risk processes breach

Policyholder
Commodity Price Behavior- Forced outages
Disintermediation

Catastrophic
 Enterprise Risk Hierarchy - Property & Casualty

Enterprise Risks

Market Risk Credit Risk Insurable Risk Operational Strategic Compliance

Interest Rate Spread Pricing Systems Process People Competitive Legal breach

Equity Price Default Reserving Information Security Inadequate training Fraud & Misconduct Sourcing Talent Regulatory breach

Foreign Exchange Migration Morbidity Implementation Risk Inadequate processes Product trends Contractual breach

Commodity Price Catastrophic Forced outages

 Models & Model Risk – Alternative definitions
Definition Source Definition of a model
SR 11-7 A quantitative method, system, or approach that applies
statistical, economic, financial, or mathematical theories,
techniques, and assumptions to process input data into
quantitative estimates.
Society of Actuaries A representation of relationships among variables,
entities, or events using statistical, financial, economic,
mathematical, or scientific concepts and equations.
Actuarial Standard of Practice #56 A simplified representation of relationships among real
world variables, entities, or events using statistical,
financial, economic, mathematical, non-quantitative, or
scientific concepts and equations.
Investopedia A model is a system, quantitative method, or approach
that relies on assumptions and economic, statistical,
mathematical, or financial theories and techniques. The
model processes data inputs into a quantitative-estimate
type of output.
Definition of model risk
Model risk occurs primarily for two reasons: (1) a model
may have fundamental errors and produce inaccurate
outputs when viewed against its design objective and
intended business uses; (2) a model may be used
incorrectly or inappropriately or there may be a
misunderstanding about its limitations and assumptions.
Model risk can be understood as the loss (economic,
reputational, etc.) arising from decisions based on flawed
or misused models.
The risk of adverse consequences resulting from reliance
on a model that does not adequately represent that which
is being modeled, or the risk of misuse or
misinterpretation.
Model risk is a type of risk that occurs when a financial
model is used to measure quantitative information such
as a firm's market risks or value transactions, and the
model fails or performs inadequately and leads to
adverse outcomes for the firm.

Regulation
Own Risk & Solvency Assessment (ORSA)
Supervision and Regulation Letter 11-7 (SR 11-7)
Actuarial Standard of Practice #56 (ASOP)
Solvency II
Regulatory Overview
Description
• Every US insurer within scope is required to file a comprehensive annual report with the NAIC.
• Incorporates elements of sound ERM practices, including use of models and tools:
• Model governance framework
• Control strategies for risk models
• Requirements and uses of economic capital models
• Importance of internal models
• Promulgated by the Federal Reserve Board and the OCC in identical form
• Designed for the investment and commercial banking industries, but widely adopted by insurers as the “gold standard”
for the development of MRM programs
• Covers model development, model implementation, model use, model validation, governance processes, governance
policies, and the related control infrastructure
• Promulgated by the American Academy of Actuaries in December 2019
• Provides guidance to actuaries when performing actuarial services with respect to designing, developing, selecting,
modifying, using, reviewing, or evaluating models
• Directive in European law that harmonizes regulation impacting insurers across the EU member states
• Allows for in-scope insurers in the EU to utilize their internal models to calculate the Solvency Capital Requirement
• Specifies that the Model Risk function should be part of a holistic Enterprise Risk Management process and function
 Steps to establish an MRM function
• Procure management buy-in on the value of establishing an MRM function – the value proposition
should extend beyond meeting regulatory requirements.

• Identify optimal organization slot for MRM – Part of an ERM group, part of an Operational Risk group, or
a standalone function.

• Develop a model risk framework.

• Develop a model inventory and a simple, easy-to-understand methodology for risk scoring models.

• Develop a management reporting layer that sits on top of the inventory and can be used to track
validation statuses, outstanding remediations, and overall compliance statistics on the firm’s model
portfolio.

• Promote an effective MRM culture with key stakeholders – model development, IT, finance, actuarial,
etc.

• Establish policies and documentation.

Model Risk Lifecycle
 Sources of Model Risk

• Incorrectly formulated input assumptions or calculation

methodologies.

• Incorrectly implemented calculation methodology or input data

uploads.

• Usage of the model’s outputs outside the acceptable range.

• Misinterpretation or improper usage of the model’s outputs in

supporting specific business decisions.
Hypothetical use cases – Model Risk events
Risk event
• A life insurer utilizes a 3rd-party Economic Scenario Generator (ESG)
model/application as an input to its cash flow projection models. The
analytics firm that manufactures the application provides automated
updates to the ESG on a quarterly basis.
• For the most recent quarterly update, the standard data quality
checks were not rigorously followed, and the update went out with
several material calculation errors.
• A monoline workers compensation insurer has a material exposure to
residential and commercial real estate in its investment portfolio.
• The insurer utilizes a simulation approach to generate a range of
return projections for these assets. The assumed return distribution is
lognormal.
• A newly hired analyst took over maintenance of the simulation model
without adequate documentation, and subsequently made calculation
errors in updating the parameters for the lognormal distribution.
• A commercial auto insurer has been a market leader in utilizing
advanced analytics like machine learning and predictive modeling to
refine the pricing of the products it sells to transportation and logistics
firms.
• After investigating the drivers for emerging losses in several states,
the firm discovered that a computational error had been made in the
development of its pricing model, leading to dramatic underpricing in
the latest annual cycle.
Mitigant/hedge
• The model development group receiving the updates should establish
its own acceptance criterion for quarterly updates that include (1)
Reviewing the testing results from the analytics team and (2)
Performing their own testing on the ESG results.
• The model risk group should examine ESG inputs for reasonableness
with every model update and revalidation.
• The model development group should establish robust written
documentation on all aspects of the model, including technical
specifics of the calculation mechanics and process flows and maps.
• Additionally, they should implement a peer review process for every
model update so that an independent review is performed of revised
results.
• Implement a robust Model Risk capability in the Enterprise Risk
Management group and require that every update to pricing models
be subject to a full model validation.
• Implement an Economic Capital model with simulation and scenario-
testing capabilities to ensure that a wide range of potential outcomes
are evaluated.
 Role that actuaries can play in model risk
• In a model development group
• Research and document alternative approaches for a particular model design.
• Develop comprehensive documentation that facilitates an efficient independent review by actuarial colleagues and
independent model risk and internal audit personnel.
• Design robust testing plans that address the model’s sensitivity to all material risk factors and input assumptions.
• In a model risk group
• Serve as the subject matter expert (SME) and validator on any insurance-specific models.
• Help to educate model risk colleagues on the economics and regulatory aspects of the firm’s insurance products.
• Assist model governance personnel in developing a robust data model to support the model inventory.
 Skills required to be an effective model risk practitioner
Skill area Specific skills
Business knowledge Product dynamics – Pricing, risk profile, marketing, regulatory, tax, etc.
Financial statement analytics
Financial strength metrics
Quantitative Financial Modeling
Statistical Inference
Simulation techniques
Predictive Analytics
Information Technology Information governance principles
Information risk management principles
Cloud computing
Transaction systems (Systems of record)
Risk Management Enterprise
Operational (including Vendor Risk)
Market & Credit
Insurable
Educational background Mathematics
Statistics
Hard sciences
Economics or Econometrics
Leadership Organizational
Communication
Project Management
 How models are used by insurers
Valuation – Statutory and GAAP reserves

Loss Reserving

Product Pricing

Business Planning

Risk and Capital Management

Underwriting automation

Catastrophe modeling – pandemics, natural disasters, terrorist attacks

Claims Analytics

Product Marketing

Fraud Analytics
Advanced Analytics – Hypothetical use cases

Mining auto claim history for insights to apply to new cases.

Use Cases
Auto insurers incorporating credit scores as an indicator
variable in premium pricing models.

Using complex models that are developed using large data sets
to more efficiently underwrite life insurance applications.

Applying pattern recognition techniques to Disability claims to

proactively identify fraudulent claims.

Develop more sophisticated pandemic risk models that take

advantage of richer internal and external data sets and greater
computational power.
 Advanced analytics – Applications to actuarial modeling

Actuarial model development

Step 1 – Step 2 – Step 3 – Step 4 – Step 5 – Step 6 – Step 7 –

Step 8 - Monitoring
Problem description Problem definition Data acquisition Data cleaning Modeling Validation Reporting

Perform variance
Level of cleaning
Data to use Quantity of data Implementation analysis (actual vs. Model selection Initial monitoring
required
expected)

Alternative model Level of pre- Complexity and Analyzing variable

Level of data quality Assumptions Ongoing monitoring
designs processing required development time importance

Implementation Assumption Partial dependency

Data sources estimation time Modeling tools
considerations plots

Level of modeling Transparency of

Business validation Testing approaches
detail design

In-scope regulatory
considerations
 COVID-19 – Impact on business outcomes
General
• As a result of the pandemic, customers are increasingly purchasing products and submitting claims online.
• This trend will likely continue after a vaccine is found. Insurers that came into the crisis with strong brand awareness and a
strong, well-established digital presence will likely fare better than those insurers that don’t have these attributes.
• Insurer’s must also adapt to increasing numbers of employees working from home. For those insurers willing to entertain
WFH arrangements, they will expand their pool of available talent.
• Insurers are increasing their focus on robust contingency planning – BCP, DRP, IM.
Property & Casualty
• Reduced auto claims due to government imposed lockdowns on customers and businesses.
• Increased business interruption claims due to business shutdowns.
• Increased trip insurance claims due to individuals cancelling planned vacations.
• Increased event cancellation claims due to lockdowns and reduced travel.
Life Insurance
• Increased volatility in investment portfolios due to capital markets volatility.
• Increased death claims due to direct impact of COVID-19 on insured populations.
• Uncertain future impacts on mortality rates due to the currently unknown long-term health impacts on COVID-19
survivors.
Health Insurance
• Increased group and individual major medical claims due primarily to longer hospital stays.
• Increased incidence and severity of Disability and Long-Term claims.
 Establishing a Model Risk program – Critical success factors
• “Tone from the Top” - must be present and strongly communicated throughout the organization.

• Gain buy-in from stakeholders – Develop a clear value proposition for Model Risk. Link positive business
outcomes to having a robust Model Risk function. Transparency is key!

• Establish a clear framework – Key definitions, roles and responsibilities, documentation requirements,
operational processes, escalation protocols, reporting, etc.

• Recruit highly skilled staff members – Model Risk roles require considerable business and technical acumen.

• Develop clear training materials – Get the message out! Make the training accessible, concise, and easy to
understand.

• Establish a model inventory – A spread sheet can work as a starting point. Expand the model metadata over
time and migrate to a GRC system if cost/benefit warranted.

• Performance management – Establish clear Key Risk Indicators (KRIs) and Key Performance Indicators (KPI) for
Model Risk.

• Risk Appetite and Risk Tolerance - Must be clearly defined and measurable.
APPENDICES
 Product hierarchy - Life, Annuity & Health insurer
Life, Annuity,
Health

Life Insurance Annuities Health

Permanent
Term Insurance Deferred Immediate Short-term
Insurance

Annual
Renewable Whole Life Universal Life Variable Major Medical
Term

Level Term Traditional Variable Fixed Rate Long-Term Care

Disability
Variable Fixed Rate Indexed
Income

Indexed
 Product hierarchy - Property & Casualty insurer
Property and
Casualty

Workers Business
Automobile General Liability Property Catastrophe Specialty
Compensation Interruption

Personal Product Liability Personal Homeowners Commercial Earthquake Directors & Officers

Commercial Renters Flood Errors & Omissions

Wind Cyber

Hurricanes Captives

Environmental
Terrorism
Liability

Major medical
 Terms of Reference
Term Definition

Data and Infrastructure - Big Data A term coined to describe data sets with sizes beyond the ability of commonly used software tools to capture, curate,
manage, and process data within a tolerable elapsed time. The term encompasses unstructured, semi-structured and
structured data…however the main focus is on unstructured data. The size threshold that defines “Big Data” is a
constantly moving target, as of 2012 ranging from a few dozen terabytes to many zettabytes of data.

Data and Infrastructure - Analytics The application of advanced computational techniques and methodologies to large data sets with the objective being
to identify, interpret, and communicate patterns in such data to decision makers to enable better decision making.

Data and Infrastructure - Artificial Intelligence A term used to describe an emerging discipline and technology within computer science, that of using computers and
other types of machines to simulate human behavior.

Data and Infrastructure – Cloud Computing A term that refers to the on-demand availability of flexible and scalable computing resources – particularly storage,
software development platforms, and distributed computational resources – without direct management of such
resources by the user.

Data and Infrastructure - Machine Learning An application of artificial intelligence that involves designing computer programs and/or algorithms with the capability
to learn and evolve independent of human beings. There are several different variants of machine learning: (1)
Supervised, (2) Unsupervised, (3) Semi-supervised, and (4) Reinforcement.

Data and Infrastructure – Natural Language Processing Refers to the technology that allows computer systems to understand and interpret human language and speech.

InfoSec - Availability Assurance that the systems responsible for delivering, storing and processing information are accessible when
needed, by those who need them.

Term
InfoSec – Confidentiality
InfoSec - Encryption
InfoSec - Integrity
InfoSec - Security Controls
Internal Models
Holdout sample/Out-of-time sample
Terms of Reference
Definition
Preserving authorized restrictions on information access and disclosure, including means for protecting personal
privacy and proprietary information.
A term used to describe different methodologies of converting plain English, readable text (known as plaintext) into
encoded, unreadable text (known as ciphertext). There are 2 elements to any encryption methodology:
• Key – The unique variable that is part of every cipher and allows the intended recipient to unencrypt the encrypted
text.
• Cipher – The algorithm or formula that is used to convert the reference text from plaintext to cipher-text.
Assurance that the information is authentic and complete. Ensuring that information can be relied upon to be
sufficiently accurate for its purpose. The term Integrity is used frequently when considering Information Security as it
represents one of the primary indicators of security (or lack of it). The integrity of data is not only whether the data is
'correct', but whether it can be trusted and relied upon.
The management, operational, and technical controls (i.e., safeguards or countermeasures) prescribed for an
information system to protect the Confidentiality, Integrity, and Availability (CIA triad) of the system and its
information.
Specified in Solvency II → “a risk measurement system developed by an insurer to analyze its overall risk position, to
quantify risks and to determine the economic capital required to meet those risks”
A data set that is not specifically used to estimate the coefficients in a model design. The data used to fit the model
comes from a different time period and is termed the training data sample. The holdout sample is then used to test
the developed model’s predictive accuracy with the same coefficients.