Modeling Decision Analysis and

Risk Management
Presented at
Bioterrorism Threat Assessment and
Risk Management Workshop

Bryan S. Ware
November 12, 2001
 Overview

• The case for risk-based decision support

• The challenge of antiterrorism risk-based
decision support
• A method to model antiterrorism risk
 The Challenge

• Capture national biological terrorism knowledge

• Transport Bio-Terrorism knowledge to
actionable decision support framework
• Give decision-makers tools to help them make
decisions in these times of crisis
 National Needs – What
We’ve Learned
• Methods to cast risk into an analytic framework that consider
complex and future threats
• Tools to facilitate job tasks within an organization
– Encourage consistent, standards-based approach to assessment and management
– Provide generation, retention, and reuse of actionable information
• Communication of risk-related information vertically and
horizontally, rapidly and efficiently
• Leverage existing knowledge bases, models, procedures, and
processes

Current practices emphasize consultants, spreadsheets,

equations, and risk analysis process diagrams
 Better Solutions (1)

• Dynamic, scalable risk management technology

Subjective evaluation equation spreadsheet Risk Influence Network

• Tools for users within the enterprise that allow them to

act on the risk information
– Analysts
– Planners
– Assessors/Auditors
– Executives/Resource Managers
 Better Solutions (2)

• Information technology architecture that links tools and

users

• Plug-in interface that allows existing internal and

external sources of data, modeling, and information to
contribute to assessment of risk

Resulting system allows THE CUSTOMER to assess,

manage, and plan for risks to your operation 24/7/365
 Risk-Based Decision-Making

Risk Assessment Risk Management Decision Analysis

• Assessment or evaluation of risk implies quantification.

– Quantification requires a systematic application of
knowledge
• Effective risk management must involve mitigation
options that are traceable back to risk assessment
• Good decisions and strategies depend on choosing
mitigation options for an entire risk portfolio.
– Bio-terrorism is not the only threat we face!
 AT Risk Management
Challenges
• A working model of AT Risk
– What is Risk? What is our language?
– What are its drivers? What does risk look like?
– Can Risk be described systematically?
• Methods for dealing with uncertainty
– How can we determine the Probability and timing of an attack?
How can we quantify anything when there are so many
unknowns and unknowables?
• Broad domain knowledge (unattainable)
– There are no anti-terrorism experts! AT Risk management
requires a broad base of knowledge (and experience) that is
unattainable in a single person (and maybe even a small group)
 AT Risk Management
Challenges
• Data availability and fusion
– The data is always available somewhere to predict and
prevent events. But, it’s seldom in the right place at the right
time to act.
• Dynamic approach to meet a dynamic threat
– Our knowledge, available data, sensitivities, and the driving
factors change continuously. So must our risk assessment
and management strategy.
• Methods to put this knowledge and framework in the
hands of lay people who must make decisions
 An AT Risk Model

Our AT Risk Model is based primarily on three

things:
1. The natural order of items in the world can be
described by hierarchy (inheritance)
2. Assets and Threats are domain concepts,
composed of world objects, that combine to
form Targets (a threat-asset pair)
3. All Targets have a similar risk structure that
can be described by a Risk Influence Network
 Risk Methodology
Implementation
Evidential Data Types and Sources Risk Influence Network (RIN)
Plausible
Plausible Tactic
Weapon

Demographic
Org Intent Likelihood
Models and Attack
Simulations History
RIN Fragment

Risk Evidence Interface

Threat Envt.
Population Risk
User Input Desirable I&W
Symbolic
Surveys Value
and Web Resources Attractive
Practical
Checklists

Distance to Accessible
Perimeter Susceptibility Consequence
Historical
Databases LOS to Asset Recognizable
Damage TARGET
Y-Axis

Press Casualties THREAT ASSET

Statistics and Architecture Mission Effect RINs are solved for

Analytic Data Labeling / Intangible each threat-asset pair
Signs Effect

More Objective More Subjective

Customer’s data &

Customer’s risk methodology
information
Backup
 Protecting Enterprise
Assets
• Risk Assessment
– Discover vulnerabilities
– Assess plausible threats
– Determine susceptible targets
– Analyze consequences

• Risk Management
– Deny threats
– Protect assets
– Control
consequences
 Site Profiler® Customization

Historical Data

Tail
ts

ored
n
r
k tio t
epo

is n

U r fa e
R liza me

In ng

se c
te in

Use
mR

ua ron

r e
E
s
Vi nvi

r Int
to

E
Cus

Object

erfa
Database

ce
rt or
A ng

o
n a in

t
ep ra
E

Plug-in
ly e

R ne
tic

e
G
Interface Bu
s els
in d
es
sR e Mo
iv
ul
es i ct
ed
Pr
 Notional Tool Layout

Planning Tools Data External/Legacy

Warehouse knowledge bases

Assessment Tools

Analysis Tools
 Example Implementation

HQ
• Policy
• Standards
Region
Region • Strategic Resource
Region Allocation
• Budgeting

• Assessment
Site 1 Site 2 Site 3
… Site n • Planning
• Resource
Allocation

Vulnerability
VA Team
Assessment
Solution Engineering
 Differentiators

• Risk management technology advantage

– Comprehensive
– Scalable
– Dynamic
• Enterprise approach
• Extensible, scalable software architecture
• Site Profiler 1.0 is complete and ready for implementation
 Summary

• Site Profiler represents a significant step

forward in risk management technology
• Site Profiler applications can be quickly
customized and deployed into an organization
• Site Profiler can be implemented in a simple
configuration, and then built upon in successive
builds
 Site Profiler Features
Risk Analysis Software Architecture IT Architecture
• Probabilistic solution of • Completely object oriented • Multi-tiered
all potential risks and database driven implementation
• Modeling of risks with • Meta-object model • Connects to other
limited knowledge or supports fundamental corporate and network
historical data threat-asset pairings and is information systems
easily modified for any
• Fusion of data from risk domain • Creates knowledge
disparate sources base for trends analysis,
• System data model KD, and lessons
• Solution of risk network contains extensible library learned
identifies key contributors, of weapons, counter-
possible countermeasures, measures, structures, and • Allows collaboration
and confidence other objects among users
• Network is broken into • Plug-in interface connects • Allows collection and
OO fragments tailored to to analytic models and dissemination of risk
specific risk components knowledge bases data and policies
throughout enterprise
• Graphical belief networks • Tools allow developers to
support SME customize GUI, DB, • Risk algorithms and
understanding and reports, and create plug-in policy can be updated
validation DBs or models as preferences or intel
 Overview

• Principles of Risk Analysis

• Government and Private Sector Tools
• Tool Comparisons
• Site Profiler Risk Analysis & Risk Management
Methodology
• Enhancements to the Site Profiler Methodology
 Principles of Risk
Analysis
• Risk analysis uses observations about what we know, to
make predictions about what we don’t know
• Risk analysis seeks to inform, not to dictate, the complex and
difficult choices among possible measures to mitigate risks
• Because decisions about risks are usually needed when
knowledge is incomplete, we must rely on informed
judgment and on models reflecting plausible interpretations
of the realities of Man & Nature
• Risk analysis relies on both basic and applied research, often
integrating information, theories, and analytic tools from a
variety of disciplines
 Sample Gov’t and
Private Sector Tools
• Government Processes and Products
– DoD Joint Vulnerability Assessment Tool
– DOE ASSESS
– CIA Analytical Risk Methodology
– FAA SAFIR
– MIL-STD-882(C)
– J-34 Antiterrorism Installation Planning Template
– DoD Joint Service Integrated Vulnerability Assessment Team
Methodology
• Commercial Products • Commercial Methodologies
– Site Profiler – Exxon Mobil Risk Methodology
– RiskWatch – KPMG Enterprise Risk Management
– The Buddy System – AON Enterprise Risk Methodology
– ALPHA – ARA Federal Security Risk
Management Process
 Mile-High View

• All risk assessment methodologies derive from

Risk = probability * consequence

= likelihood * severity

• Implementations look different depending upon:

– Domain of application (results in different terminology for same things -
intrusion vs. attack)
– What does the user “know” (i.e. has control over specifying or calculating)
– The answer the analysis is intended to provide
– Technology that is used to implement the methodology (i.e. stubby pencil
vs. simulation)
 Common Functions

• Assess Threat All tools and

• Characterize Assets methodologies share
these functions – there
• Create Threat-Asset Pairs are very few
• Assess Vulnerabilities exceptions
(definitions vary widely)
• Calculate & Display Risk Data
• Rate Risk Acceptability
• Apply Countermeasures &
Iterate
 Enterprise Risk Mgmt

• Encode risk management business rules into risk

management architecture
• Embed tuned risk management methodology into tools for
users within the enterprise
– Analysts
– Planners
– Assessors/Auditors
– Executives/Resource Managers
• Link tools and users through an Enterprise architecture
• Link to outside organizations and outside sources of
information
 An Asset is anything that is considered important from
the user or customer’s perspective or attractive as a
Target from the enemy’s perspective
In JVAT, Assets are broken into five
Domain Asset categories: Building Assets, People
Model Assets, Area Assets, Infrastructure
Building Asset
Assets, and Other Special Assets
(Headquarters)

Building Object

Structure

Physical
World
Model World Model

Asset
Domain objects require programming for
customization, but World objects can be
created easily through database updates.
 Domain Threats are composed of a Terrorist
Model and a Weapon System. The Threat
Threat Domain Object composes a threat
(Truck Bomb) from World Model objects. JVAT
supports 15 types of Threats with
Weapon hundreds of weapon system
Terrorist
System
configurations.

Delivery
Weapon
System
Threat
Vehicle Bomb Truck TNT
Terrorist
Attack

Terrorist Land
Org. (foe) Weapon
Attack Vehicle

Agent Events Physical

World World
Model
 Threat

Asset Threat

Target

Attack

Threat Vector

Site Profiler™ creates Targets by combining

Threats and Assets. Each Target pair can
have one or more Attacks and multiple Threat
Vectors for each attack. Each Threat Vector Asset
has consequences associated with it.
 Consequences
For most Threats, Consequences are determined
by calculating the number of casualties and the
damage to Assets and the resulting effect on
Mission. Consequences are calculated using
Plug-in Analytical models.

JVAT includes blast

and CBRN models
including HPAC,
BEEM, BELT, and
CELT.
 Domain AT Functions are composed of
Model Organizations, Procedures, and
AT Function Counter-Measures that can be
(Check Point)
used to reduce the risk of a
Threat-Asset pair.
Organization Procedure CM’s

Random
Gate
Inspection
Military Police Guard Guard

ID check Barrier

Org. (friend) Procedure CM Object

Agent Events Physical

Check
Point
World World
Model
Creating an Asset
Asset

Building Asset

Attributes:
• Asset Type
• Labeling
• Population
• Mission Role
• Ease of Access.
• Prox. To assets
• Recognizability
• Symbolic
Domain and Object Model Creation
Building Asset

Attributes:
• Asset Type
• Labeling
• Population
Desirability
• Mission Role Asset Type
Attack History

• Ease of Access. Proximity to Recognizability

• Prox. To assets
other Assets
Population
Press
Demography

• Recognizability Demograph X
Asset Type Architecture

• Symbolic
is present
Labeling
Time sensitivity
Mission Role of Mission

Recoverability

Asset: Building Asset

Symbolic Value

Building Object

Attributes:
• Number of stories
• Percentage of glass
• Frame construction
• Panel type
• Glass type
• Underground parking
garage
 High-Level RIN
Accessibility
Susceptibility
Recognizability
Asset Threat
Practicality
Target
Attractiveness
Desirability
Target

Mission Effect
Risk
Intangible
Consequences
Legend Effect

Casualties
Asset
Plausible
Weapon
Target
Plausible Threat
Tactic Likelihood
Threat
Organizational
Intent
 Risk Influence Network (RIN)
• Domain objects contain the attributes that influence
the risk of a given event
• These attributes are represented by probability
distributions
• These attributes combine to form a Bayesian belief
network representation of domain knowledge
• This RIN can be solved hierarchically…as more data
becomes available, more of the network nodes are
used for calculation
• The network is broken into fragments that are tied to
Domain Objects and supported by World Objects
 Bayesian Network Basics

Conditional Probability Bayes Rule

P(A | B) = X => Given the event B, the P(B | A) = P(A | B) P(B) => The Probability of B
Probability of A is X P(A) given A is equal to the
Probability of A given B
times the Probability of
B, divided by the
Probability of A

Parameters of Interest and States Conditional Probability

Table
A = The chance of rain b1 b2 b3
States = High (a1), Medium (a2), Low (a3) a1 0.8 0.15 0.05
B = The chance you’ll carry an umbrella
a2 0.2 0.6 0.2
States = High (b1), Medium (b2), Low (b3)

a3 0.1 0.2 0.7

 Simple Bayesian Network
Prior Probability Table gives initial
belief in A independent of B

a1 a2 a3
Parameter of Interest,
also called a “Node” A 0.3 0.5 0.2

Arrow indicates a
casual influence, b1 b2 b3
arrow tip indicates
a1 0.8 0.15 0.05
direction of influence B
(A influences B) 0.2 0.6 0.2
a2

a3 0.1 0.2 0.7

Conditional Probability Table

gives belief in B given A
 Site Profiler Physical View
Installation
Security Other View Servers
Applications

Planner Windows GUI Policy Synchonizer

Manager

Interface Language)
XUL (XML User
- Reads Polices - Sends Objects

Analyst - Executes Global

Policies
- Receives Objects

User Interface

ODMG 3.0
HTML
Manager Database
Presentation Descriptions Workflow
Server Server
- Presennts Screen
representation info Object
User Interface - Presents Workflow
based on user type Information
Events
Surveyor PALM
Presentation
XOL (XML Object

Server Processed
Translation
Language)

Object Data Engine

- Generates Events
- Processes Event
Scripts
Object - Policy Monitors

Modifications Model Script

Results Events
Plugin Interface
COM/DCOM

Decision Analytic I/O Module I/O Module

Support Models (XML) (Pager)
Engines (e.g. BELT,
(e.g. BNSE) CELT,
BEEM, etc.)
 1.X Layer Diagram

Site Profiler

Site Profiler
Computational Report
Application 3D Runtime GUI Engine Persistent Objects Plugin Interface
Engine Generator
Component

Interface
Open GL Custom API ODBC ADO/OLE DB Custom Interfaces
Standard

Interface
API COM
Mechanism

External Model I/O Analysis

Bayes Engine MS SQL Server MS Word Plugin Plugin Plugin
Application
 Threat Alert Form
Threat Alert Form Specific Terrorist Organization
Name:
Credibility of this alert: High Individual(s):
Use the following sections to define non-specific Have made statement: Yes

I&W and/or specific target/threat data Credibility of statement: Very Credible

Credibility of organization: Credible
Indications and Warnings
Previous weapon use: Favored
The following has been observed:
Previous tactic use: Used
Training
Surveillance
Plausible Tactics
Recruiting
Prepositioning Tactic Type Credibility
Propaganda Vehicle Bombing Credible
Material Acquisition
Other Suspicious Activity Assassination Very Credible

Biological Very Credible

Specific Targets
New Tactic
City:
State: Plausible Weapons
Country:
Weapon Type Credibility
Target Type Credibility
Sedan Bomb Credible
Building Tanker Truck Bomb Not Credible
Headquarters Credible
Handgun Very Credible
Office Building Not Credible
People Anthrax IED Not Credible

VIP Very Credible

New Weapon
Civilian Not Credible
 Site Profiler Risk IDEF

CM Standards,
Asset Analysis Intelligence Vulnerability Risk Prioritization Risk Reduction Cost
Standards Standards Standards Standards Standards Standards

Define Define Analyze Target Analyze Analyze Risk Prioritized Cost / Risk Reduction
Assets + Prioritized Risks +
Assets Assets Threats Vulnerability Pairings + Risk Reduction Benefit Plan + CM
Threats Risks CM
CMs Measures Upgrades Analysis Requests

Plan Plan Plan Plan Report

Builder Builder Builder Plan Plan Report
Builder Generator Builder Generator
Builder
Report Report Report
Generator Report
Generator Generator
Generator
Profiler Intelligence Analytic
Analytic
Model Models
Models
Profiler
Profiler
Bayes Engine
Interfaces
High Level Asset Model

ASSET

INFRA- OTHER
BUILDINGS PEOPLE AREAS MATERIAL
STRUCTURE
ASSETS
 Target Model

A Target is the
association of
one or more
assets and zero
or more threats.

Threat_Target_Pair 1+
THREAT ASSET

TARGET
High Level Threat Model

THREAT

ID
Desc
Name
Active Y/N

AGENT

WEAPON
SYSTEM

DELIVERY
SYSTEM

WEAPON
 Countermeasures

PHYSICAL AGENT PROCEDURE

COUNTER-
MEASURE

COUNTER- Countermeasure_effectiveness
THREAT
MEASURE

EFFECTIVENESS

Denial Effectiveness
Deterrence Effectivness
Delay Effectiveness
Detection Effectiveness
Mitigation Effectiveness
Interruption Effectiveness
Response Effectiveness

Operation:
 High Level World Model
Space
Land Vehicle Air vehicle Water craft Sensor Barrier
vehicle

Miscellany Counter- Structure

Area Weapon Vehicle Natural
measure

PHYSICAL

Military Civilian Environment

Human Organization

AGENT EVENT

SITE PROFILER