Lean RW Final.
qxp 4/16/2007 9:56 AM Page 225
Basic Elements of a Comprehensive
Root Cause Analysis Program
By: Mark Galley, ThinkReliability, Houston, TX
The purpose of incident investigation, problem solving,
troubleshooting and root cause analysis is to determine why a
particular issue occurred and to identify specific actions to be taken
to prevent negative consequences from occurring. A complete
investigation program defines the overall approach for a single
person, group, site or even an entire company. Five of the basic
elements are; 1 - methodology, 2 - measurements, 3- the process, 4 -
facilitation, documentation and storage and 5 – regular review of the
program. The principles that are part of any single incident
investigation will be the same for a complete investigation program,
only the scope of the implementation will change. A company wide
approach can be started with one person in one group solving
problems effectively. It can then be scaled to a department, site and
division. The principles are the same, but the scale regarding the
number of people and level of coordination will change. Each of the
five elements of a complete investigation program will be summarized
in the next sections.
1. RCA Methodology
There are four basic characteristics of an effective root cause
analysis (RCA) methodology to be effective. 1 - It must be based on
cause-and-effect relationships supported with evidence, 2 – it must
be scalable, 3 – it must be simple and 4 – it must connect back to
work processes.
The basis of any sound investigation methodology is an extreme
bias for the cause-and-effect principle supported with evidence.
Regardless of the type of problem worked, a safety issue, an
equipment failure, a service outage or a people issue, every incident
breaks down into cause-and-effect relationships. The more an
organization aligns with the basic lessons of cause-and-effect the
more effective all investigations, troubleshooting and communications
will be. The opposite of this is also true - the more an organization
makes decisions based on speculation, conjecture, hearsay and
opinion while disregarding facts and data the more ineffective that
organization will be.
Because cause-and-effect is a principle it can be applied
universally to any size and type of issue. The amount of detail
required in a large investigation is obviously more than in a small
incident, but the method shouldn’t change. Some organization use
one method for their large issues and a different investigation method
for their smaller issues. Large or small, the lessons of the cause-and-
effect principle do not change – only the level of detail. Likewise
some companies use one methodology for safety issues and different
approach for production or equipment issues. An actual incident
investigation usually involves several different departments. Having a
single, principle-based approach helps show the interconnectedness
/ Reliability Basics
between different groups. Changing investigation methods from one
group to another can create discontinuities that cause groups to
remain disconnected and separate. It’s easier to establish alignment
across groups when the focus is on the principles, not technique.
There may be subtle variations in different group approaches, but the
principles remain consistent for the people participating and leading
investigations. The more the investigation can clearly and accurately
reflect that actual incident the better it is for the investigators,
participants, managers and executives. When big issues, mid-size
incidents and day-to-day troubleshooting of problems are worked in a
similar manner it establishes a problem solving culture in the
organization. A consistent approach for managers and front-line
personnel helps align problem solving to meet the overall goals.
The simpler the root cause analysis method the better the results.
Simple doesn’t not mean incomplete or inadequate. By simple we
mean fundamental. An organization that focuses on the basics doesn’t
have to be creative with the language of cause-and-effect. Most
companies are inadvertently drawn to unnecessary adjectives when
using cause-and-effect analysis. There is no need for differentiating
types of causes during an investigation. The focus in the analysis
step should be on the causes, not the type of cause. Any discussion
about the types of causes (the adjectives) during an investigation can
take the groups focus away from evidence-based causes by wasting
time on arbitrary labels. Companies have plenty of acronyms and
terminology for their particular industry that simplifies the
communication when people in the same field are talking about an
issue. It is important to remember however how differently people
see problems. What one person calls a problem is what another calls
a symptom. Trying to add terms to an investigation or problem
solving method can create disagreements and miscommunications.
One of the most effective ways to summarize this point is that the
investigation methodology has got to be simple enough that people
will use it. As soon as it’s confusing, frustrating or takes too long it is
less likely to be used.
The last point is the importance of connecting every investigation
back to the work processes that played a role in creating the original
situation. Before any incident occurs there were already work
processes in place. People were already doing some tasks when the
incident occurred. The actions items from a particular investigation
are implemented back in the work processes. Every action item from
an investigation creates some addition, modification or refinement in
the original work process. Even though a group wants to improve the
way they investigate issues they don’t want to do more
investigations. The ideal number of investigations a group can have
is zero. No group want’s the investigation. What they really want is
to get better at executing the process without errors or difficulties.
The objective isn’t to conduct more and more investigations - the
2007 Conference Proceedings 225
Lean RW Final.qxp 4/16/2007 9:56 AM Page 226
objective is to conduct fewer and fewer because the processes are
being performed effectively. The entire concept of reliability is to be
less reactive and more proactive. This bias for connecting every
incident back to work process keeps people’s focus, not on fire
fighting and investigations, but on being proactive by improving
processes.
2. Measurements - Criteria
Any time an improvement is made there is a baseline that we
measure from. Someone knows that they lost 5 pounds after one
month of exercise because they knew what they weighed one month
ago. They had to have a baseline for weight and the starting date.
Establishing a baseline in an investigation program is required to
determine the specific improvements.
“What do you want to improve?” is one of the first questions when
beginning an investigation program. The same approach applies to
looking at a single incident or failure. If a piece of equipment has
failed 6 times in the past two years we want to reduce those failures.
This would be a fairly straightforward measurement. When
establishing an investigation program for a particular group a Top 10
list may be identified. The list would have the top ten items that
impact that organization’s goals. For an investigation program across
an entire company there would be numerous baselines. It’s
important to understand what the baselines are for safety rates,
environmental issues, customer problems, production outages or
delays and excessive material and labor costs.
These baselines are the basis for measuring improvements. A
baseline is just a starting point (in performance) and a date. Without
a baseline the gains cannot be validated with evidence. Measuring
any deviation from the goals of the organization drives the entire root
cause analysis program. Organizations should be investigating those
things that have the biggest impact on the overall goals. The key
performance measurements or indicators (KPIs) that an organization
uses reflect overall goals of that organization. The overall goals are
the ideal state for that organization. As an example, the overall goal
for safety in an industrial company is zero injuries. If the goal is zero
injuries then any injury is a problem. Clearly some injuries are bigger
than others, but the goals truly dictate and prioritize all of the
problems. An organization doesn’t have one overall goal they have
several. Typically they are safety, environmental, customer,
production-schedule and materials and labor. Any impact on any one
of these overall goals, that is any deviation from zero, is what people
call a “problem.”
Depending on the impact to the overall goals some issues may be
investigated to a much more detailed level than others. This is
commonly referred to as severity or importance. The severity of an
incident depends on how it impacts the goals. A large impact to the
organization’s overall goals is a relatively larger issue. A potential
impact or near miss may also be a high severity because of the
potential risk of the issue. A near miss fatality with no injuries (no
one was injured, but it could have been a fatality) will have a
226 2007 Conference Proceedings
relatively large amount of detail compared to a minor injury that
actually occurred.
Defining the overall goals (ideal state) up front in an investigation
program aligns the entire focus on the program. Once the goals are
defined the problems and their severity is simply a matter of checking
the impact or deviation from the goals (this is also referred to as the
gap). Some companies use the measurements of overall equipment
effectiveness (OEE), asset utilization (AU), or maintenance spending
as ways of providing direction when determining the performance of
equipment or operating units. The performance measurements can
also be done by area, by incident type or by work process. These
different types of measurements are helpful, but it’s important to
remember that they all provide some relative measurement based on
one or more of the overall goals. An investigation program should be
tied directly to the overall goals of the organization so the small, mid-
size and large issues can clearly be defined to give direction to
individuals and groups. Every group that conducts investigations
should be measuring against the overall goals of their organization.
3. Investigation Process - Roles
Just as detailed steps can be defined for any work process when
operating a piece of equipment, the detailed steps of the complete
investigation process must be defined for it to be established and
maintained. This investigation process is not an explanation of the
investigation methodology, but an overall view of all parts of the
investigation program. It must define the specific roles within the
program and what each one is responsible for. It also needs to
include the measurement criteria for what makes something an
incident and defines its severity by type of issue. The severity helps
dictate how detailed the investigation needs to be and the people in
the organization that may need to be involved.
The investigation process must have an overall owner. For a
given group, site or company there must be a person who owns the
investigation process. There must also be an owner for each of the
investigations. The owner of the investigation is not necessarily the
person that solves the incident being worked, but they do organize
and collect all of the information for that issue. The incident owner
is typically called the facilitator but there is more to the
investigation than the investigation facilitation. The owner of the
incident gets involved as soon as an incident occurs. They may be
at the scene of the incident immediately to help contain the incident
and preserve any information that would otherwise be lost. The
owner coordinates the meetings (times and dates), opens and leads
the investigation and ensures that all information is captured and
documented. The training requirements and skills must also be
defined for this incident owner.
The investigation process not only defines the role of the owner
(facilitator or lead investigator), but it defines the role of the people
that will be participating in the investigation. What should people
expect when participating in an investigation and what information
(evidence) do they need to provide? What training is required or
Lean RW Final.qxp 4/16/2007 9:56 AM Page 227
suggested for the people that will be participating? The roles of the
managers will also be defined. What should the managers be
expecting and asking for as the output an investigation.
The management of action items must also be defined in the
investigation process. Not only do we need to know who the action
items are assigned to but we also need to know how they will be
signed off or completed as well as the process for past due action
items to ensure their completion. The action item process must also
consider any update in paper work or process documentation
because of any changes being made (management of change). The
effectiveness of the solutions and whether or not the solutions
delivered the planned results must be part of the follow-up. If this
follow-up on the solutions is not part of a defined process it will not
necessarily occur.
The investigation process not only defines all of the key roles, but
how the entire investigation program functions. The investigation
process is a training tool for anyone participating in or leading an
investigation and needs to be updated as any improvements are made.
4. Facilitation, Documentation
and Storage
Facilitating an incident investigation and documenting it are
connected to the storage of the incident. All three of these pieces
are grouped together in this section. Each one will be broken down
with its related information.
The facilitator is the owner of the incident investigation. Facilitate
is defined as “to make easier.” One of the easiest ways to think of
the investigation facilitator is the person that collects and organizes
all of the information related to an issue. The facilitator would be
familiar with not only the investigation methodology, but also the
entire investigation process. The facilitator is not necessarily the
problem solver. The facilitator understands how cause-and-effect fit
together, but the participants in the investigation are the ones
providing the details and evidence.
It obviously helps if the investigation facilitator is comfortable in
front of a group leading a review of the incident. There are some
fundamental people skills that a facilitator would ideally posses.
Keeping the groups focused, keeping the investigation moving along,
being mindful of the time and the investigation progress are all part of
the facilitator’s role. There are facilitator skills that have to do with
people and meeting effectiveness. And there are investigation
facilitator skills that have to do with capturing cause-and-effect
relationships, providing supporting evidence and soliciting ideas and
solutions from the group. Ideally both aspects of the facilitation are
part of an effective incident investigation. The facilitator needs to be
someone who wants the role. The level of interest by the facilitator
will be reflected in the facilitation.
The facilitator may talk to people one-on-one or they may get a
few people together in a group. It may take one meeting, it may take
/ Reliability Basics
a dozen. The facilitator may collect a significant amount of
preliminary information, photos, diagrams, a timeline, and statements
of fact, before the first meeting is ever held. Participants in the
investigation may arrive to find a preliminary packet of information at
each seat with the cause-and-effect relationships already displayed
on one wall in the meeting room so that the group can edit as
necessary. In other cases, the facilitator may not have any
information together because of the nature of the incident and the
timing of the investigation. In this case the facilitator starts from
scratch with the group and builds all of the related information in the
investigation into a coherent picture.
The documentation aspect has a couple of parts: what to
document and how its documented. What to document in an
incident usually consists of a timeline (sequence of events) an outline
that captures the impact on the overall goals, the causes with
evidence and any related diagrams, photos, statements and
information. The action items (solutions) are always captured in an
action plan with owners and due dates assigned. This information
can be put together in a hardcopy report or an electronic file.
How the information is documented is a function of the skill and
background of the facilitator. Ideally the information is captured in an
organized, useable format as it is collected in the investigation.
Meaning, the causes, evidence and statements are captured as it is
shared the first time. This can be done using pen and paper, on a dry
erase board, on chart paper or electronically (using various software
tools such as Microsoft Excel). This aspect of the documentation
refers to the speed of the investigation. The investigation can take
one hour or three hours depending on how it is documented during
the investigation. The faster the documentation is, the faster the
investigation is. Facilitator skills have a significant impact on this
aspect of investigating an incident.
The storage of the incident can be done in a variety of ways
depending on how it was documented and the tools that organization
has available. A simple way is to put a hardcopy of the report in a
file folder to keep all of the information together, but this is only
available to people that can get to the hardcopy folder. It can also be
stored electronically using various tools, the simplest, least
expensive, most flexible and most readily available is Microsoft Excel.
Storing the incident electronically makes it much more accessible to
other people across a network or even via email. This is where a
database of incidents can be very helpful for retrieving and tracking
incidents. A database of just the incident is relatively simple and
provides powerful sorting capability. The incident can be
documented electronically using Excel, but the database keeps the
entire Excel workbook as an attachment to the incident record within
the database. It makes searching for incidents very easy and keeps
the software platform simple.
One way to store the information from recurring incidents is by
building a visual of all the past failure modes. A Cumulative Cause
Map™ can be created containing all the incidents that have occurred
for that piece of equipment or unit. These Cumulative Cause Maps
2007 Conference Proceedings 227
Lean RW Final.qxp 4/16/2007 9:56 AM Page 228

are built electronically, but because they typically have a large specifics that must be defined for each organization, it is at least a
amount of information on them they can plotted on wide paper and foundational framework with which to build a comprehensive
placed a wall. The wall plot can be used as a summary of all the investigation program. Investigating and solving problems is part of
failure modes that have been identified to this point. Some may think of every organization in every industry. The more effective individuals
it as troubleshooting guide. These Cumulative Cause Maps are another and groups become at solving incidents the more effective the
way to capture and use a significant amount of information in a simple organization becomes at meeting its overall goals. Improving not only
format – people’s experience can be shared visually rather than stored the how the investigation was done, but also the speed, clarity and
in their head. Many of the process maps for a given work process are results of the investigation is an ongoing effort that should become
also stored on a wall just like the Cumulative Cause Maps. Anyone with part of the culture within an organization.
new, different, conflicting or additional information should add it to the
existing map by writing directly on it. This approach provides a point
for collecting everything we know that caused or causes a system or
piece of equipment to fail. In concept, this approach is similar to a
failure modes effect analysis (FMEA), but it is captured visually. These
visual documentation methods don’t necessarily replace a report, but
they can complement it and help create a learning organization that
shares organizational knowledge more easily.

5. Regular Program Review

The last element of a complete investigation program is the regular
review of the process. This step is ongoing because the investigation
program must be evolutionary. A complete investigation program can
be defined by one person within their group. If the investigations are
effective and incidents are reduced people will copy what works
especially if it’s relatively straightforward.

The program review involves all elements of the program. The

questions that need to be asked at least quarterly are “What works
well and what needs to be improved in the methodology, the
measurements, the investigation process, and in the facilitation,
documentation and storage?”

When the investigation program is one person it’s fairly simple to

get feedback. The facilitator who is probably also the owner of the
investigation process can ask the people in the investigation what
they thought of the approach and what they would like to see
improved. As the program grows to encompass a site or entire
business the number of people involved will increase and the
feedback from participants can become scattered. It may take a little
longer to collect all of the feedback, but the program will
fundamentally be the same for working a single issue as it will be for
a comprehensive investigation approach. A complete investigation
program is only as effective as any single investigation. Regular
review is important with a willingness to incorporate any
improvements back into the definition of the process. Always
connecting the investigation program to the overall goals of the
organization will also help to keep everyone on track with what’s
important and why. An investigation program should measure the
results of the investigations (the effectiveness of solutions and
improvements in performance measures), not the activity of
conducting investigations.

There are several different aspects to the fives elements covered in

this paper. While they don’t provide the detailed blueprint with the

228 2007 Conference Proceedings