How to Audit

the Difficult Areas

of a
Quality Management
System

Whittington & Associates, LLC

242 Highlands Drive, Woodstock, GA 30188
www.WhittingtonAssociates.com
800-404-7585 or 770-517-7944

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 1

 Introduction

Some parts of a quality management system are

more difficult for auditors to assess:
1. Undocumented Process
2. Legal Requirements
3. Resource Management
4. Continual Improvement
5. Preventive Action
6. Internal Audits
7. Process Effectiveness
ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 2
 Introduction

To discuss how to best audit these areas, we first

have to clearly understand the requirements.
Then, we need to remember that auditors collect
evidence from these primary sources:
• Interviews (statements from responsible persons)
• Observations (demonstrations and operations)
• Documents (plans, procedures, and instructions)
• Records (past practices as proof of conformity)

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 3

 1. Undocumented Process

• Documents required by ISO 9001 (per 4.2.1.a-c)

– Quality Policy; Quality Objectives; Quality Manual
– Document Control and Record Control Procedures
– Internal Audit and Nonconformity Control Procedures
– Corrective Action and Preventive Action Procedures
• And, documents needed for effective planning,
operation, and control of processes (per 4.2.1.d)
• Work instructions are optional (unless operating
under industry sector scheme like ISO/TS 16949)

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 4

 Undocumented Process

• How audit if requirements aren’t documented?

• Ask the process owner to describe the process
• Use manager statement as requirement source
• Carefully watch the process being performed
• See if documents actually exist at work place
• Examine records to match practices to intent
• Write nonconformity report if find a discrepancy
• Action doesn’t have to include adding document
• Avoid suggesting expanded text just for auditor

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 5

 2. Legal Requirements

Does ISO 9001 address legal requirements? Yes.

• 5.1.a - Top management must communicate
importance of meeting customer, as well as,
statutory and regulatory requirements
• 7.2.1.c - Organization must determine statutory
and regulatory requirements for product
• 7.3.2.b - Inputs to design must include applicable
statutory and regulatory requirements
These legal requirements are for quality system
and product, not health, safety, or environment.
ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 6
 Legal Requirements

• Identify applicable legal requirements for area

• Ask legal staff, contract group, and audited area
• Ensure requirements are available for reference
• See if monitor for new or changed requirements
• Request evidence of conformity to requirements
• Issue NC if legal requirements not considered
• Issue NC if area in violation of legal requirement
• Help area to comply with statutes and regulations
Requirements: customer, company, standard, legal
ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 7
 3. Resource Management

• ISO 9001, clause 6.1, requires organization to

determine and provide resources needed to:
– Implement and maintain quality system
– Continually improve system effectiveness
– Enhance customer satisfaction
(by meeting customer requirements)
• Resources include: equipment, facilities, people,
supporting services, work environment, suppliers,
information, natural resources, and finances

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 8

 Resource Management

• Are resources being identified, planned, made

available, used, monitored, and changed?
• Assessing performance to evaluate resources?
• Don’t audit in isolation; verify performance results
• Interview top management; examine the evidence
• Don’t make subjective judgments on adequacy
• Limit role to judging effectiveness of resources
• Avoid being placed in middle of resource dispute
• Issue NC on “problem” due to lack of resources
ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 9
 4. Continual Improvement

Continual Improvement is the “recurring activity to

increase the ability to fulfill requirements.”
Clause 8.5.1 requires continual improvement of the
effectiveness of QMS by use of quality policy, quality
objectives, audit results, data analysis, corrective
action, preventive action, and management review.
• Effectiveness is “extent to which planned activities
are realized and planned results achieved.”
• Quality Policy, 5.3, must include a commitment to
continual improvement of effectiveness of QMS
ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 10
 Continual Improvement

• Are continual improvement projects identified?

(beyond corrective and preventive actions)
• How were rates of improvement determined?
• Are plans approved and resources allocated?
• Keyed to requirements and satisfying customers?
• Compare performance results to quality targets
• Not a nonconformity if targets are not being met
• If not met, analyzing why and revising the plan?
• Unable to improve in all areas at once (prioritize)
ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 11
 5. Preventive Action

“The action to eliminate the cause of a potential

nonconformity or other undesirable situation.”
• ISO 9001 requires documented PA procedure
• Combined CA and PA procedure is acceptable
• Determine action to eliminate causes of potential
nonconformities to prevent their occurrence
• Action must be appropriate to effects of problem
• Evaluate need; determine and implement action
• Keep records of results; review actions taken
ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 12
 Preventive Action

• Understand PA versus Correction versus CA

• How are potential nonconformities identified?
• Best time is early in product cycle, e.g., FMEA
• Look at the nonconformity trends and patterns
• Examining warning signals for out-of-control?
• Look at records of preventive actions and results
• Verify action effectively prevented potential NC
• Goal of PA is avoiding possible NC (status quo)

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 13

 6. Internal Audits

Audit: a systematic, independent, and documented

process for obtaining audit evidence and evaluating
it objectively to determine the extent to which audit
criteria are fulfilled.
Conducted at planned intervals to determine if the
quality management system conforms to:
– Planned arrangements
– ISO 9001 requirements
– Organization requirements
and is “effectively” implemented and maintained.
ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 14
 Internal Audits

Describe audit process in documented procedure.

Plan the audit program to consider:
– Status and importance of processes and areas
– Results of previous audits
Define criteria, scope, frequency, and methods.
Select auditors, and conduct audits, to ensure:
– Objectivity
– Impartiality
Do not audit your own work.

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 15

 Internal Audits

• Are scheduled audits conducted as planned?

• Are all functional areas and shifts being audited?
• Are the auditors competent and independent?
• Do audit reports show procedure being followed?
• Is schedule adjusted based on past audit results?
• Is more audit attention given to high risk areas?
• Do audits examine conformity and effectiveness?
• Are all requirement types used as audit criteria?
• Are audits conducted using “process approach”?

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 16

 Internal Audits

• Are weaknesses in poorly performing processes

being identified by audits?
• Are NCs spotted before found in external audits?
• Are OIs being identified by internal auditors?
• Are CAs properly verified before audit closure?
• Are audit program objectives set, tracked, met?
• What is auditee and management feedback?
• Have any OIs been identified for audit process?

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 17

 7. Process Effectiveness

Audit focus usually on conformity, not effectiveness.

Requirement is to audit effectiveness of processes.
Process is a set of interrelated or interacting
activities which transform inputs into outputs.
Process Approach is the systematic identification
and management of processes, and particularly
their interactions.
Effectiveness = extent to which planned activities
are realized and planned results achieved.

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 18

 Turtle Diagram

Resources Resources
What? Who?

R R
E E
Q Q
U U
I I
INPUT R R OUTPUT
Receive Deliver
What?
E PROCESS E
what?
M M
E E
N N
T T
S S

Methods Measures
How Done? What Results?

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 19

 Process Effectiveness

• View system as set of integrated processes

• Understand their interfaces and interactions
• Adopt the process approach for your audits
• Add value by looking at more than conformity
• Evaluate linked processes for “effectiveness”
• Verify their controls and identify process risks
• Determine any opportunities for improvement
• Promote process view through audit methods

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 20

 Summary

Difficult areas to audit:

1. Undocumented Process
2. Legal Requirements
3. Resource Management
4. Continual Improvement
5. Preventive Action
6. Internal Audits
7. Process Effectiveness
Questions about auditing these or other areas?

ASQ - March 2007 © 2006 Whittington & Associates, LLC Slide 21