27/01/2018 Deviation Management - Why Have We Not Solved the Problem?

| IVT

Search

GXP
Deviation Management - Why Have We Not Solved the Problem?

By David W. Husman Jan 23, 2018 8:00 am PST

Abstract

Deviations occur across the pharmaceutical and device industries nearly every day. Companies have
spent hundreds of hours and thousands of dollars to investigate deviations, identify their causes and
implement corrective actions to prevent their recurrence. Yet with all of these investments, personnel
and our management can continue to be frustrated as the same problems continue to recur.

FDA and other regulators routinely request to see lists of deviations, non-conformances, CAPAs,
complaints, and out of specification investigations during inspections. For nearly 20 years, failure in
these systems has been in the top 5 of all observations issued. So what are we doing wrong? This
article attempts to explore the issues and provide some possible solutions and paths forward to
eliminate the problems routinely documented as deviations within our manufacturing and testing
environments. Although the examples focus on U.S. FDA, the problems and solutions are applicable
worldwide.

I. Introduction

Numerous articles, seminars, regulations, and guidance documents have been generated in an
attempt to address the problems observed by regulators, auditors and quality assurance
departments. Despite all of this attention and the abundance of available resources to address
problems, the FDA and other regulators continue to cite failures with deviations and non-
conformances, CAPAs, complaints, and out of specifications. These failures to adequately address
management of deviations, their investigations, and lack of analysis exploring reasons for
occurrences, have become a bane to the regulated industry. THe industry spends extensive time in
dealing with backlogs, recurring problems, audit findings and regulatory citations.

Too often those in the industry have a variety of definitions, concepts and tools that are frequently
used interchangeably without ensuring that they are talking about the same or different things. One of
my colleagues frequently stated that we often are in violent agreement when we thought we were on
opposite sides of an argument.

To avoid confusion, this article will provide a brief list of definitions of common deviation terms,
followed by some tools repeatedly found in use in typical deviation management systems, some
frequently cited observations from regulators, characteristics of common deviation systems and this
author’s suggestions for why the citations continue, and finally some ideas for eliminating these
problems in the future.

II. Definitions

A. Correction/containment: steps taken as soon as possible to correct or stop the deviation from
continuing and contain the potential impact. Examples: Fixing the specific instance of a record error,

http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 1/9
27/01/2018 Deviation Management - Why Have We Not Solved the Problem? | IVT

turning off the water after a pipe burst, quarantine of suspected product.

B. Corrective Action: steps taken to address the underlying cause of a deviation to ensure the
deviation does not recur in any part of operation. Note the usage – to correct and prevent Recurrence
of a problem.

C. Deviation: Departures from laws, standards, manufacturer’s instructions, policies, procedures,

specifications, or accepted practices that are not discovered and corrected prior to, or by, the next
critical control point in the system. Note: this is a somewhat more narrow view as items discovered
and corrected prior to a critical control point are also deviations, but in this author’s opinion do not
make sense to manage within the deviation system).

D. Out of Specification (OOS): A test result that is a variance from an established material or product
specification.

E. Out of Trend/Tolerance (OOT): A test result that is atypical from established historical norms for
this equipment or material.

F. Preventive Action: steps taken to ensure a deviation never occurs. These actions are typically part
of a continuous process improvement process. Note the usage – to prevent Occurrence of a problem.

G. Root Cause: The basic reason(s) a deviation has occurred (why a deviation occurred).

III. Typical Root Cause Analysis tools

There are a myriad of tools in use throughout the industry but some of the most common are listed
below with a brief description.

A. Retesting

Although not thought of as a Root Cause Analysis tool, this process is typically used to identify
whether an initial OOS/OOT result was valid and if so, identify a cause for its occurrence.

B. Fishbone /Ishikawa Diagram/5 M’s

There are a variety of variations on Kaoru Ishikawa’s Fishbone diagram in use to document cause
and effect of problems. Although the labels vary, in principle they fall into evaluating the following
categories for their effect on a problem: personnel, procedures, materials, environment, machines,
and measurements. The principal goal of this approach is to expand the scope of thinking.

C. 5 Why’s

The basis for the 5 Why’s approach is to understand the cause of actions by delving deeper below
the symptoms. This process helps teams recognize a broad network of problem causes and their
relationships and drives problem solvers to successively pursue causes for additional causes until
underlying “root” causes become evident.

D. Fault Tree Analysis

This process is designed to show linkages between issues and to explore each linkage individually. It
helps teams get to solutions to problems that may have more than one cause contribution.

E. Affinity Diagrams

This process organizes large numbers of ideas into their natural relationships. This method taps the
team’s creativity and intuition and frequently is used to narrow solutions.

IV. Examples of objections (from Turbo EIR, FDA.gov)

http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 2/9
27/01/2018 Deviation Management - Why Have We Not Solved the Problem? | IVT

Next we are going to look at common examples of findings / objections from regulators. Although this
presentation focuses on FDA data which points out common reoccurring issues, it is also applicable
to other regulators findings across the world.

A. There are no written procedures for production and process controls designed to assure that the
drug products have the identity, strength, quality, and purity they purport or are represented to
possess. Cited 21 CFR 211.100(a) Written Procedures; deviations.

B. There is a failure to thoroughly review [any unexplained discrepancy] [the failure of a batch or any
of its components to meet any of its specifications] whether or not the batch has been already
distributed. Cited 21 CFR 211.192 Production Record Review

C. The following although admittedly somewhat difficult to read is a consolidation of findings from
FDA.gov’s turbo EIR program that are commonly linked to deviation management failures: Deviations
from written [specifications] [standards] [sampling plans] [test procedures] [laboratory mechanisms]
are not [recorded] [justified]; Established [specifications] [standards] [sampling plans] [test
procedures] [laboratory control mechanisms] are not [followed] [documented at the time of
performance]; The establishment of [specifications] [standards] [sampling plans] [test procedures]
[laboratory control mechanisms] including any changes thereto, are not [drafted by the appropriate
organizational unit][reviewed and approved by the quality control unit].

D. Procedures for corrective and preventive action have not been adequately established. Cited 21
CFR 820.100(a) Corrective and Preventive action

E. Corrective and preventive action activities and/or results have not been adequately documented.
Cited 21 CFR 820.100(b) Corrective and Preventive action

V. Costs Associated with Management of Deviations

While we all recognize that cost should not be a primary driver for quality decisions, in practice all too
often companies fail to recognize all of the costs involved with investigation of deviations and how
some of these costs could be eliminated with a different approach to deviation managment. The
primary costs include routine Investigation and Investment in people and tools to better manage
deviation investigations and records.

A. Routine Investigation

Investigation costs arise from a variety of sources including but not limited to:

Time spent investigating, analyzing, developing CAPA and implementing actions to eliminate or
prevent a problem
Time spent reviewing and approving deviation reports
Costs of disposition of non-conforming materials
Costs related to implementing CAPA
Costs for remediation of deviation systems including additional training
Lost opportunities (loss of reputation, product shortages, impacts on production schedules,
etc.)

The costs for investigation typically can run between several hundred to several thousand dollars per
occurrence of a deviation. A simple calculation often used (# of hours x # of people x $ loaded
employment rate (hourly salary + benefits) addresses just the cost of management of the deviation
investigation and ensuring records are appropriately reviewed and closed. It should be noted however
that there are also the containment costs related to the management of the non-conforming material
(quarantine/storage), the management of impacted equipment, utilities and facilities (ensuring they
are offline/not in use until issues resolved) management of impacted personnel, and additional
management of production schedules.

http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 3/9
27/01/2018 Deviation Management - Why Have We Not Solved the Problem? | IVT

Additionally, there are the costs related to remediation e.g. creation of new procedures,
replacement/repair of equipment, utilities and facilities, validation/revalidation of equipment, methods
and production processes, additional testing, drug/device shortages, recalls, lost reputation, etc. As
you can quickly see the costs of even one deviation can quickly add up into the tens of thousands.

B. Investment
1. Technology

The industry has invested many tens of thousands of dollars in electronic tools including but not
limited to: Sparta Systems – Trackwise®, Master Control-Quality Management System (QMS) and
Pilgrim Quality – SmartSolve®. Each of these tools provides data platforms for capturing, tracking
and trending deviations and the actions resulting from them.

Typically these systems are designed with various forms for collecting information in searchable fields
and allowing multiple attachments. Although excellent tools, each of these systems has been used
across the regulated industry to varying degrees of success. The mentality here seems to be if I just
automate the deviation management process, the problems will go away.

2. Personnel

The industry has spent many tens of thousands of dollars in providing training and other support to
personnel in an attempt to address the continuing issues of recurring deviations. These include but
are not limited to: In-house training, attendance at internal and external seminars and extensive use
of consultants. The mentality seems to be let’s train / re-train / hire consultants to make the problem
go away.

VI. So why is it not working?

With all of the investment in time, money and energy, why do we continue to receive citations, not
resolve our problems, and eliminate the recurrence of deviations? Below are some ideas for some of
the fundamental causes that lead to our recurring issues.

A. Thinking our job is to manage deviations/CAPAs

Many companies believe that it is their job to create a deviation system that manages problems
encountered at their companies. They miss the concept that their real job is to eliminate the
problems, create documentation that demonstrates they eliminated the problems and prevented their
recurrence.

FDA reports that more than 95% of all deviations they encounter during inspections are associated
with recurring issues and are corrective in nature with less than 5% being preventive in nature.
Numerous FDA officials and consultants have stated in multiple conference presentations that the
goal is 95% of deviation investigations should be preventive with only 5% being corrective in nature.

B. Failure to tell a good story

At the end of the day, if we do not tell a story that is clear, concrete (factually based), concise,
complete, shows that the problem was corrected, and all documentation closed we get no credit for
any of the work and effort put worth in our deviation management documentation. It is a well-known
axiom that if it isn’t written down, it didn’t happen. But if others can’t understand what you did, you
really are just wasting your time. Our goal should not be to write long epistles but rather to clearly
demonstrate both to ourselves and to others that we identified a problem and promptly took actions to
investigate, understand and resolve it.

C. Confusing and conflicting responsibilities

http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 4/9
27/01/2018 Deviation Management - Why Have We Not Solved the Problem? | IVT

Most Deviation Management systems are owned and operated by the Quality organization.
Deviations however, are not routinely originating in the Quality organization but rather in other parts of
the operations. The challenge immediately then arises – who is responsible for evaluating and
resolving the deviations – the owner of the deviation system, the originator of the deviation or
someone else? Most systems in the industry are set up with an immediate notification of the quality
organization but from that point the variety and effectiveness vary. Some Quality organizations
believe it is their responsibility to determine if an occurrence is a deviation and if so, what should be
done, and frequently who is assigned. Others see their role as micromanaging the investigations, still
others as wordsmiths of the documentation, and still others as final arbiter of the conclusions.
However, too frequently, these approaches result in Quality being the policeman and owner of the
problems. As the owner, the quality organization is then accountable for ensuring they are fixed while
the creator of the problem is exonerated.

Other Quality organizations see their role as primarily or even exclusively oversight and frequently
distance themselves from the day to day execution of the management of the individual deviation.
Operational personnel frequently execute the investigation and analysis of the issues always looking
over their shoulder for what is Quality going to think about what they did. When documentation is
provided to the Quality organization there is frequent back and forth (with the concomitant delays)
until the Quality group agrees with the final document.

D. Lack of Knowledge

1. Failure to understand the systems being evaluated

Too often while there is a recognition that a process does or doesn’t work, there is typically a general
lack of knowledge for how the process is supposed to work. That is, how the equipment, personnel,
materials, methods, environment and measurements come together to create a process that ensures
it works consistently. Without this understanding, when a process fails, staff more often than not
proceed to implement solutions in an attempt to get the process working without truly understanding
whether they have returned the process to its original designed state. The end result more often than
not is that the corrective actions are more Band-Aids that cause other problems rather than cures.

2. Failure to Gather Facts of the incident

The basic facts of the incident help us to fully understand what happened on this particular instance
of the execution of a process and allow us to compare and contrast these facts with other executions
of the process where there were no incidents. Too often, our deviation process is 1) notify Quality, 2)
issue a form, 3) assign a risk classification, 4) decide what the root cause is and 5) come up with
some CAPAs in an attempt to fix the issue. We skip gathering facts about what was actually going on
at the time the problem occurred. More often than not, this results in problems being open for
extended periods of time while staff determine what to do.

When specific facts including but not limited to time of occurrence, personnel present (did we speak
to them), location of occurrence, what was going on before during and after the occurrence, how long
did it take to gather the information, can we repeat the error, was anything moved before you
evaluated it, etc. result in lost opportunities for understanding, and all lead to incomplete and more
often than not incorrect conclusions. Additionally failure to gather facts often results in incorrect or
incomplete assessment of the risks the issue posed and directly leads to either too few resources
assigned or too many to evaluate the issue.

3. Failure to link problems to the systems responsible for making sure they didn’t occur in the
first place

Too often we look at deviations as individual ocurrences or random acts of problems and more often
than not concluding one or more individuals caused the problem. What we frequently miss in our
evaluations is that we designed systems (e.g. production management, materials management,
http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 5/9
27/01/2018 Deviation Management - Why Have We Not Solved the Problem? | IVT

equipment management, analytical testing, etc.) to execute our processes that took into account that
individuals were involved when they were designed. If a problem occurred, we typically never
consider whether we didn’t design the system correctly to ensure it was capable of overcoming or
preventing errors when personnel engaged them.

E. Using right tools in wrong ways

Below are three common tools used in the deviation management process, but in this authors opinion
are used the wrong way.

1. Expecting Automated Systems to Define the Deviation Management Process

As noted above, Sparta Systems – Trackwise®, Master Control-Quality Management System (QMS)
or Pilgrim Quality – SmartSolve® and others are frequently used to manage deviation data. As data
management tools they are very effective in capturing problems, tracking the closure times, and who
provided the closure. But as in any automated systems, there are examples where these tools have
been employed effectively and where these tools have been employed to the detriment of companies
as they document repeated deviations and the failures to eliminate them.

Pharmaceutical and Medical Device companies frequently have failed to have well defined deviation
management systems that they are /were attempting to automate. Too often, organizations stop short
by allowing the tool to define their deviation/CAPA process rather than procedurally defining their own
process and implementing a tool configured to meet the needs and requirements of their process.
While commercially available automated systems do a good job in capturing data and allowing for
tracking and trending deviations, their out of the box systems are more effective at defining processes
that manage data than they are at defining processes for investigating, analyzing and correcting
problems. The tools typically provide locations for the outputs of these processes without defining the
execution of these processes.

An additional problem routinely identified with this approach is that user requirements have not been
adequately defined for either the tool or the process and therefore the resulting validation efforts are
inadequate/incomplete.

Regulators have pointed out that companies are using the tools in the wrong way by the observations
they write. Several User Requirement failures routinely encountered by auditors and regulators,
demonstrate how using the right tool in the wrong way will not. Each of these are described below:

1. Absence of or incompletely defined outputs

Failure to have defined a user requirement for formatted outputs of the electronic records. Companies
are frequently unable to generate complete and legible paper representations of the electronic
documentation or to provide true and accurate copies of the electronic documentation.

2. Insufficient size and number of electronic fields

Limiting field size and number without ensuring that necessary information that clearly describes a
step often leads to either missing information or ‘see attached’ statements. These both limit the
usefulness of these fields as searchable tools during trend analyses.

3. Undefined and unlinked attachments

Attachments should assist in providing facts and telling the story about the issue, how it was
investigated and resolved. Where these documents are not coherently linked to the deviation, or
where their organization is unclear, their purpose is no longer evident.

4. Poorly defined trending categories

http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 6/9
27/01/2018 Deviation Management - Why Have We Not Solved the Problem? | IVT

The key to a good deviation management program is the ability to trend issues to ensure that they
have been eliminated. The use of too broad or too narrow of categories and tracking by root causes
often obscures issues that are repetitive or suggests linkage of issues that have nothing to do with
each other.

2. Fishbone /Ishikawa Diagram/5 M’s

While Fishbone /Ishikawa Diagram/5 M tools are important analytical tools for ensuring investigations
consider all aspects of issues that could be involved in a problem, too often this tool is used more in
the investigation process as an elimination tool. As such the investigations typically assume a defined
set of causes and proceed to gather information that eliminates those causes from consideration,
often without thoroughly evaluating each of the items. More often than not, these investigations
eliminate all ‘causes’ except for personnel and then attribute the cause of the problem to be human
error.

The intent of the tool is to appropriately and thoroughly evaluate each item for its contribution to a
problem to avoid oversimplifying issues. Some examples of over simplification:

When evaluating procedures/methods, the focus is more on does the procedure and or steps
exist rather than are there specific steps that clearly provide instruction for the consistent
execution of processes and are they effective?
When evaluating equipment, the focus is on whether there were any objections or notifications
documented in logs rather than do all parameters (including but not limited to speed, pressure,
alarms, sound during operation, etc) demonstrate the equipment was performing as designed?
When evaluating measurements the focus is on whether values are within or outside of
specification rather than does the variability of the measurement impact the conclusions?

3. 5 Why’s

As noted above, the purpose of the 5 why approach is to delve deeper into a problem to determine if
there are multiple actionable causes supported by the data. However, our focus often results in trying
to get to the ‘one right answer’ or the easiest solution to fix ending in a lack of depth in our
evaluations. Conversely, the process results in a never ending theoretical evaluation of all possible
events without considering whether they are probable or whether the current situation conditions
support the identified cause.

VII. So What Can We do differently?

A. Proactively evaluate systems and remediate where necessary

As noted above, the expectation of regulators is that the vast majority of our efforts should be in
ensuring problems never occur in the first place rather than in dealing with problems after they arise.

Systems such as calibration, maintenance, training, qualification, standard operating procedures, test
methods, etc. should be designed to ensure activities are suitable for their intended use and operate
consistently. Where we have a lack of detailed knowledge and understanding, efforts should be
undertaken to obtain this knowledge. Where necessary addition of steps to error proof processes
should be undertaken. Frequently these error proofing involve more specific details in set up, take
down and cleaning of equipment.

As noted above, equipment, personnel, materials, methods, environment and measurements play
important parts in consistent process execution and therefore are all potentially contributing parts of a
problem. When a problem occurs, it is critical to evaluate the various systems we have in place for
ensuring these problems do not occur. At a minimum we want to capture the occurrence so that trend
analysis can be used to recognize whether one or more systems is having multiple failures. For
example, when an equipment failure occurs, we should evaluate the occurrence and timing of

http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 7/9
27/01/2018 Deviation Management - Why Have We Not Solved the Problem? | IVT

preventive maintenance, calibration, cleaning, lubrication, qualification and measurements to ensure

those systems were functioning appropriately. That is, not just is it within the designated windows, but
did the activities occur recently or are they near deadlines and are those windows appropriate.

B. Establish/Reestablish the appropriate roles and responsibilities

Investigations are best executed with teams made up of persons knowledgeable about the issue as
well as with persons that provide the independent eye. The Quality organization has specifically
required roles of oversight and must ensure that it maintains this role. Oversight should ensure the
process executes as designed and documentation is generated to reflect that activity. However,
Quality organizations must also have staff that partners with the process owners to ensure that
investigations remain objective, fact based, and do not devolve into opinions. At the end of the day
the company as a whole must be proud of the documentation that is created, irrespective of the
underlying issue that was encountered.

C. Use tools to document facts and to evaluate the gathered facts for what are they telling you.
Determine real root causes.

Facts are the most critical factor in successful investigations. Timely gathering of facts at the location
where the problem occurred ensures that they represent what was actually going on at the time this
issue arose. Delays in fact gathering and attempting to create the investigation from behind a desk
frequently lead to gaps in our information (don’t remember, disassembled equipment, missing
records, moved items, etc.) about the issue at hand. Answering the basics of who, what, when, and
how an issue arose allows us to determine why it occurred. Once we know why it occurred, we can
determine what actions are required to ensure it does not recur.

Well-designed tools both prompt the investigator to document gathered facts as well as facilitate
capturing how the conclusions were reached.

D. Link Problems to systems and remediate systems as necessary

For more than 10 years FDA and other regulators have been talking about quality systems in various
regulations and guidance documents. Many of these internationally recognized systems are
described in ICH Q7-Q10 for pharmaceutical systems and in various ISO guidances, such as ISO
13485 for device systems. Well-functioning systems ensure problems do not occur or when the do
the systems are in place to minimize their impact on patients. When problems occur, it is critical to
successful companies that they also evaluate their systems to ensure they are or continue to function
to eliminate problems and minimize any problems that occur. Where problems point to one or more
systems not performing as required, these systems must be remediated.

E. Tell a good story

Well written deviation/CAPA documents coherently describe the sequence of events that occurred,
and both the conclusions that were drawn as well as how those conclusions were reached. All
conclusions should be supported by facts.

Your goal is always that any reviewer reaches the exact same conclusion as you do. A well written
story that is coherent, comprehensive yet concise and clearly shows that issues were identified, they
were promptly investigated, that appropriate actions were taken to prevent impact to patients and that
the issue was resolved in a timely manner is fundamental in eliminating citations from auditors and
regulators.

VIII. Conclusion

While the industry has spent much time and effort in dealing with deviations, it has very little to show
for the investment. This article attempted to point out some of the common mistakes being made
across the industry. By eliminating these issues and focusing more on the investigation process and
http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 8/9
27/01/2018 Deviation Management - Why Have We Not Solved the Problem? | IVT

capturing a coherent story, we will be better able to demonstrate that we thoroughly understand what
happened and thus the actions we have taken to eliminate the problems are both appropriate and
effective.

PRINTER-FRIENDLY VERSION

http://www.ivtnetwork.com/article/deviation-management-why-have-we-not-solved-problem 9/9