A Perspective on Risk

Analysis for the GMP

Initiative
H. Gregg Claycamp, Ph.D., CHP
Center for Veterinary Medicine
Office of New Animal Drug Evaluation
hclaycam@cvm.fda.gov
April 2003

1
Outline
„ The Premise and Questions
„ Basics of Risk Analysis
„ Possible Stages of Risk Assessments for GMP
Initiative
„ Risk Ranking Model for a GMP Initiative?
„ Pilot Scale
„ Conclusions
The opinions and ideas presented here are those of the author and do not
represent policy or opinion of the FDA. This material is intended for
discussion purposes only.

2
 Premise: Links Among Process (GMP)
Risks and Patient Risks are Lost
cGMP Patient

Quality (Patient)
Inspection Risk Æ

Factors
Processes

Correlation?

RISK Æ RISK Æ

3
Goal: Re-Link cGMP Risks with Actual
Risks to the Patient
cGMP Patient
Inspection Risk Æ

Quality (Patient)
Processes

Factors

RISK Æ RISK Æ

4
The Question…
„ Can Risk Management theory, tools, practice and
philosophy be employed to re-link risks to the patient
with the risks identified, perceived or otherwise
implicated in the product quality regulatory process?

„ How can we share a common language about risk, risk

management and science-based decision making so that
we can focus on developing a high-quality risk
management model for product quality?

5
Getting Started…
„ What theories, tools and lessons learned in risk
analysis can help address these questions?
„ Given the need for a significant shift in the
approach to risk management, how do we begin
the change process?
Are there off-the-shelf models and tools that
might be used, i.e., at a pilot-scale?
What kinds of RM processes can be used to
foster changes needed both the regulatory
and industrial spheres?

6
Basic Risk Analysis

7
Starting with the Some Basics
Risk is intuitive and familiar to everyone, yet few
among us define risk carefully and formally
enough for complex risk analysis.

8
Risk = “exposure to a chance of loss”
(or, Risk = “chance of losing something
we value”)

Risk = Hazard x Exposure

RiskConsequence = Hazard x Exposure
9
Contemporary Risk Analysis

„ Includes four major activities:

Hazard Identification
Risk Assessment
Risk Management
Risk Communication

10
Risk Assessment Precedes Risk
Management
„ Risk assessment is not a single process, but “a
systematic approach to organizing and analysing
scientific knowledge and information.” NRC (1994)

„ Various paradigms exist for the execution of a

risk assessment in public health; however, all
paradigms have in common fundamental
principles.

11
Risk Assessment Asks:

„ What can go wrong?

„ What is the likelihood it would go wrong?
„ What are the consequences?

12
Risk Management Asks:

„ What can be done?

„ What options are available?
„ What are risk trade-offs in terms of risks,
benefits and costs?
„ What are the impacts of current
management decisions on future options?

13
Risk Analysis in a Democracy
„ Risk assessments provide the “facts” for risk
analysis.
Risk
Risk
Risk

Worst Æ
Risk
Risk Risk
Risk
Risk Risk Assessment Risk
Risk
Risk
Risk Risk
Risk

14
Risk Analysis in a Democracy
„ The risk management decisions about which
risks to manage are value-laden decisions.

Risk Management
Risk Values Risk

Rank Æ
Risk Risk
Worst Æ

Risk Risk
Risk
Management Risk
Risk Risk
Risk
Costs Risk
Risk
Risk Risk
15
 Translating Risk Analytic Paradigms
Risk
RiskAnalysis
Analysis

Hazard Identification
a p pplylyaa
a n geeto
t o adpel. .
g
p lelecchharnaryymmoodel
Risk Assessment i m p
AAssim temp•pooWhatrar can go wrong?
o n
ccont e m • What can go wrong?
•• Release Assessment
GMP Failures •• What
Whatarearethe theconsequences?
consequences?
•• Exposure
Exposure Assessment
Assessment •• What
Whatisisthe
thelikelihood
likelihoodthat
thatitit
•• Consequence would
wouldgogowrong?
Consequence Assessment
Assessment wrong?
•• Risk
Risk Estimation
Estimation •• What
Whatcan
canbe
bedone?
done?
•• What
Whatare
arethe
thetrade-offs
trade-offsin
interms
terms
of
ofcosts,
costs,benefits
benefitsand
andrisks?
Risk Management risks?
•• What
Whatisisthe
theimpact
impactof
ofdecisions
decisions
Risk Communication on
onfuture
futureRMRMoptions?
options?

16
Possible Stages of Risk
Assessment for the
GMP Initiative

17
Hazard Identification
„ What can go wrong?
Identify hazards: events
Identify hazardous agents (chemical, biological,
physical)
„ How severe are the potential consequences?
Given the event occurs, is the consequence
catastrophic? Mildly annoying?
„ How likely are the events to occur?
Essentially a crude risk estimate for initial prioritization
purposes.

18
Exposure Assessment
„ Release Assessment: How “much” of the
hazardous event occurs?
Example: Does a “non-sterile” event involve 1
or 10,000 vials?
„ Pathway analysis: If the hazardous event
occurs, what pathways are there that expose
humans to the hazard?
„ Extent of exposure: If a hazardous event occurs,
how many people are potentially exposed?
19
GMP Failure (Release) Assessment
„ How frequent are the identified GMP events
(hazards)?
„ Boundary of release? Process line, plant,
warehouse, distributor?
„ Release rates (“GMP Faults”) are obtained in
fault tree assessments, empirically, historical
data, expert analyses.
Example: FMEA

20
Consequence Assessment*
„ Given exposure to the hazardous event/agent,
what is the likelihood of harm under a pre-
defined endpoint?
Endpoint examples:
„ Death
„ Illness
„ Worry
„ OAI

*A.K.A. “Dose-Response Assessment” (see next slide)

21
Consequence Assessment

100%
persons who become ill
Proportion of exposed

50%

ED50
0%

Quantity of contamination (“non-sterility”)

i.e., in “bacteria counts per vial”

22
Qualitative Consequence Assessment
Relative Effect/Impact

High

Medium Quantitative
relationships known
in few cases

Low

Low Medium High

(Exposure or Dose Metric)

23
Risk Estimation
„ Bring together the information about
the hazard,
the extent of exposure to the hazard,
the consequences of exposures, and then
estimate the risk.

„ Includes a critical analysis of uncertainty in

both the data and risk assessment models.

24
Uncertainties in Risk Assessment

Knowledge Variability
•Data •Temporal

•Parameters •Spatial

•Model •Inter-individual

25
Conceptual Models for
RM in GMP Initiative

26
The cGMP Risk Management Problem
„ Diverse GMP failure (hazards) are identified.
„ Wide-ranging risk (= chance that exposure to the
hazard will result in harm [adverse outcome]).
How can we objectively rank
„ Wide-ranging consequences (death to worry).
“apples and oranges” among the
„ Quantitative“potatoes
risk analysis hazard-by-hazard
and beans?” too
vast an undertaking.
„ Ranking of risks for re-linking worst GMP risks
with worst health risks, etc.

27
 Bulb
Fault Trees for Fails each process?

No Glass Filament Vacuum

electricity Broken Broken Leak

Power Plant Power Line Connector

Impurities Vibrations
Fails Fails Corroded

Wind Breaks Tree Breaks

Line Line
28
Faults Magnified N-fold for a Simple
Manufacturing Process

29
Decision Analyses for Each Hazard Multiplies
Complexity!

e.g.,

30
Solution? A Multifactor Approach to
GMP Risk Management
„ Multifactor methods already exist.
„ Some tools (software) already developed.
„ Appropriately-scaled approach to
the question,
the data quality,
the nature of the decision, and
the understanding of the overall process.
31
State the Assumptions
„ E.g., assume that health risks were linked to
GMP “compliance risks” previously, i.e., the
historical basis of regulation.
Historically based assumption:
↑compliance Æ ↓Health risk
↑quality
„ Given the assumption, can GMP “compliance
risk” be modeled as a surrogate of health risk?

32
Identify the GMP Failures (Hazards)
„ What can go wrong?
„ Top level organization of hazards:
Health | Compliance | Resources | Sociopolitical
„ Second level (detail) organization:
Sterility (microbial contamination)
Dose (formulation)
Toxicity (chemical contamination)
Physical hazards (physical contamination/defect)
„ Fine detail: “risk factor” event descriptors.
33
Sort the Hazards/Risks by Major
Categories
„ Start with assumptions.
„ State questions to be answered.
„ Sort under the questions.
„ Re-sort if new patterns emerge.

For example, (next slide)…

34
Organizing a Multi-factorial Risk Model
Risk

Health Compliance Resource

Socio-
Political
…
Death VAI Human Public

Chronic Illness OAI Inspection $ Industry

Acute Illness The Hill

Mental Health

35
Focused Multi-factorial Risk Model

Risk

Health Compliance

Death VAI
Example
compliance
Chronic Illness OAI risk endpoints
Example
health risk
endpoints Acute Illness

Mental Health

36
Risk factors for a given endpoint…

Health Compliance

Death OAI

Sterility Sterility

Lyophilization Lyophilization

Final Sterility Final Sterility

etc. … etc. …
…

37
Estimate the Prevalence
„ The prevalence of inspection findings for a
given type of event are initial estimates of
probabilities necessary for risk
management modeling.
„ Failure analysis “in plant.”
„ Failure in compliance inspections.
„ Human adverse events.

38
For each hazard…

Health Probability of Occurrence

Very Very
Endpoint Low Low Medium High High
Death Medium Medium High High High

Chronic
Low Medium Medium High High
Illness
Acute
Low Low Medium Medium High
Illness

Worry Low Low Low Medium Medium

39
The modeler’s view… (for example)

Health Probability of Occurrence

Very Very
Endpoint Low Low Medium High High
Death 5 4 3 2 1

Chronic
6 5 4 3 2
Illness
Acute
7 6 5 4 3
Illness

Worry 8 7 6 5 4

40
For each hazard…

Compliance Prior History of Actions

Never Few Average Some Many
Endpoint Violations Viol. Viol. Viol. Viol.

OAI Medium Medium High High High

VAI Low Low Medium High High

Other? Low Low Low Medium High

41
Scoring, then prioritize multiple hazards

Probability of Occurrence

Ve
ry
Ver
Probability of Occurrence
y
Scored and Prioritized
Endpo Lo Medi Hi Hig
int w Ve
Low um gh h Ver

1. GMP Fault A
ry y
Endpo
Me Lo Probability
Medi of
HiOccurrence
Hig
int diu Med
w Low Hi
um Hig
gh h
Death High
ium Ve gh h Ver
m
Me ry y
Endpo Med
Lo Hi Probability
Medi Hig
Hi of Occurrence
Hig
Death diu High

2. GMP Fault T
Chron ium gh h
Lo intMed
m Medi
w Hi
Low Hig
um gh h
ic
w ium um gh Ve h Ver
Illness
Chron Me ry y
Lo Med Med Hi Medi Hig Hi
ic Death diuEndpo Medi
Me
Hi
Lo High Hig Hig
w ium ium
um gh Low gh h um h
Acute Lo
Illness Low m int
Medi
diu
w Hig gh h

3. GMP Fault C
Illness w um h
m
Chron Me Me
LoDeath Med Med Hig
Hi High Hig Hi Hig
Acute ic Lo Low
Medi diu Medi
diu ium
Illness w w Me
ium
um um gh
h h gh h
Lo Illness m Med m
Worry Low Low diu
w ium
m

4. GMP Fault D
Chron Me Med Me Medi Hig
Acute
Lo Loic Lo Medi Med Hi Hig
Worry Low Low diu ium diu um h
Illness
w w Illness Low w um ium gh h
m m

Me Medi Me
LoAcute Lo Med diu Hig
Low Low

5. GMP Fault X
Worry Low diu um
w Illness w ium m h
m

Me
Lo Med
Worry Low Low diu
w ium
m

6. GMP Fault M

42
Risk Ranking & Filtering Model
Compliance
(Risk Ranking and Filtering)
Scored and Prioritized
Health Under Multiple Criteria
1. GMP Fault M
2. GMP Fault T
3. GMP Fault C
Other… 4. GMP Fault D
5. GMP Fault X
6. GMP Fault A

43
Risk Analysis Cycle
Start
cGMP/Compliance
Inspections
Risk Assessment Risk Management

Assessments
Work Planning
(Data Bases)

Multi-Factorial RiskRanking
Risk Ranking
Risk Model andFiltering
and Filtering

44
Pilot Scale?

45
Example Approach to Build RRF List*
Risk Estimator
Respondents
x GMP Faults Database

Analysis

- Risk Managers
- GMP Experts
- Sr. Managers Risk Ranking
- Industry

46
Fold into cGMP Model

Worst
Risk Management
Risk Ranking “Cut-Off”

Best
(Table) • Budget
• Risk Tolerance
• Benefit-Costs
• Stake holders

47
Conclusions
„ Risk Assessment provides a process for
organizing information in support of risk-based
decision making.
„ Risk assessment is one of the tools available for
Risk Management, the activity in which the
options for controlling risks are examined in light
of costs, benefits and risk trade-offs.
„ Multifactor Risk Ranking and filtering approach
might be robust enough to employ in the GMP
Initiative.
48