System/ Clinical Audit

Mebratu Massebo, MHA,PHD

 Definition
• Clinical audit is the systematic and critical
analysis of the quality of clinical care.
• This includes the procedures used for
diagnosis and treatment, the associated
use of resources and the effect of care on
the outcome and quality of life for the
patient.
• Clinical Governance = improving
standards
Clinical audit

Clinical effectiveness

Risk management

Use of information

Education and training

Staff/staff management

Patient/public involvement
The 7 pillars of clinical governance
 Definition – less formal
• Taking note of what we do
• Learning from it
• Changing it if necessary
• With the aim of improving care
 Why do It?
• Development of professional education and self regulation
• Improvement of quality of patient care
• Increasing accountability
• Improvement of motivation and teamwork
• Aiding in the assessment of needs
• As a stimulus to research
• Clinical audit aims to lead to an improvement in the quality of
service providing:-
• improved care of patients
• enhanced professionalism of staff
• efficient use of resources
• aid to continuing education
• aid to administration
• accountability to those outside the profession
 Fundamental Principles
• All about improving patient care
• Should be seen as part of day to day
practice
• Developing a critical eye on what we are
doing
• Trying to improve things all the time
 Clinical Audit

Why should you do clinical audit?

 Critical analysis of your own skills and ways of working

(self improvement)
 Critical look at the system that delivers care to your
patients (i.e. your environment)
 Can make a real difference and change things for the
better
 The Audit Cycle
What Should Be Happening

What Is Happening?

What changes are needed

 Cost cutting exercise

Data collection A piece of

for its own sake research/study

Audit Is Not……

A computer or
A witch-hunt number crunching
exercise
 Comparison Between Clinical Research
& Clinical Audit

Characteristic Clinical Research Clinical Audit

Purpose PROVE IMPROVE
Character Scientific Inquiry Systematic Measurement
Function Sets Standards Compares Standards
Sample Size Statistical Significance Commitment to Act

Bases for
Testing Hypothesis Standard of Practice
Measurement
Methods New Treatment No New Treatment
Outcome Increased Knowledge Improved Practice
 The Audit Cycle
Select a Topic
(what should be happening)

Observe
Implement Current
Change Practice
(What is
happening?)

Compare with Standard/Guideline

(What changes are needed?)
 Clinical Audits Focus
• Structure audits: focus on what is needed to deliver
care. equipment and facilities. A structure audit asks right
resources/equipment to conduct work.

• Process audits: focus on what is done to patients in terms

of the delivery of care. processes and procedures. A process
audit ask if the patient given the right treatment at the right
time

• Outcome audits: focus on what you expect to happen to

patients as a result of their treatment and care. This may be in
terms of auditing changes in the patient’s current and future
health status. An outcome audit may ask if patients have
attained the appropriate outcome.
 Making Audit Easier – Avoid the
Blocks
• BEFORE YOU START
• Time – big audits can eat up time in an already busy schedule, so :
• Keep it simple and small
• Look at one or two criteria
• Engage the whole team – otherwise it will be difficult! Is the team
ready? (Enthusiasm, wanting to improve)
• WHEN YOU START
• Delegate & Share the workload – involve others
• Make life easier – use computers to do the laborious stuff (patient
searches)
• Use protocols / standards already laid by others (why re-invent the
wheel?)
• Be careful of data collection – choose a topic which does not entail
too much data collection to the extent it becomes exhaustive with
subsequent loss of enthusiasm
 Clinical Audit

Choosing a topic

• Relevant
• Potential benefit
– To patient or to the organization
• Must have a known standard.
• Own area of interest.
• Areas you can influence.
 Some Ideas
• You can do an Audit of
• Structure ie facilities being provided
• Eg waiting times, availability of staff, record keeping (all patient
records should have a summary card), equipment
• Process ie what was done to the patient eg referrals, prescribing,
investigations
• Aspirin post MI, BP measurements 5 yearly in those aged 20-65
• Outcome ie result for the patient
• Eg patient satisfaction, patients with high BP aged between 20-35
should have a diastolic below 90mmHg within the first year of
treatment
• high risk practices (significant event audits) eg pneumococcal
vaccines in splenectomised patients, are significant events being
acted upon?
• The outcome is the ideal indicator for care but the most difficult to
measure.
 Clinical Audit

Planning audit

• Decide what to audit (Select topic & write objectives )

• Choose Criteria /Standards / Guidelines
• Methodology (sample / timeframe / data source /
prospective / retrospective)
• Involve stakeholders - & if possible patient
• Objectives to measures
– Feature (s) of quality
– % to meet standard
– Exceptions
– Definitions
– Clarity about data source
 Choosing a Topic
• Remember, topic should be important :
• Chronic Disease Management eg referrals
or use of lab services (INR’s in warfarin)
• Preventative Care eg childhood imms,
Cervical Cytology
• Prescribing eg post MI, PPI’s (cost issue)
 Examples
• Ways of spotting audit topics                examples
• Important clinical events                         admissions for asthma
• “Significant events”                                 patient died of MI – no
record of smoking history or BP
• Patients' complaints                                too long to get an
appointment
• Observation                                             no system for ensuring bag
drugs up to date
• Observations of staff                               patient on Warfarin not had
INR for 6 months
• NICE subjects                                          post-MI patients on Mx
 Criteria
• = yardsticks
• “An audit criterion is a specific statement of what
should be happening.”
• A statement which
• A) defines a measurable item of health care
which
• B) can be used to assess quality
• KEYPOINT
Criteria should be explicit. You must
demonstrate evidence for justifying them
(literature search, Evidence Based!).
 Criteria – KEY POINTS
• Ensure that the criterion is measurable –
• ·            “asthmatics should have had yearly PFs” is
difficult to measure (how many years will you go back?);
• ·            “asthmatics should have had a PF recorded in
the past year” is more practical.
• Don’t try to audit too many criteria at once – one or two
will keep you busy enough.
• Try filling in the gaps of the following phrase to set your
audit criterion:
• “All patients with xxxxx should have had a xxxxx in the
last xxxxx.”
 Criteria
• "All eligible women aged 25-65 should
have had a cervical smear in the last 5
years."
• “All asthmatics should have had a Peak
Flow recorded in the past year.”
• “All drugs in our emergency’ bags should
be in-date.”
 Standards
• “An audit standard is a minimum level of
acceptable performance for that criterion.”
• Make sure the standard is directly related
to the criterion, also :-
• Should include a suitable timeframe
 Clinical Audit

Audit standard
The standard should reflect the clinical and
medico-legal importance of the criterion

In the example,
“80% of women should have had a cervical smear”

“But of those who had an abnormal smear, 100%

should have had action taken”
 Standards
• → Examples:
• "At least 80% of eligible women aged 25-65 should have
had a cervical smear in the last 5 years."
• “At least 60% of asthmatics should have had a Peak
Flow recorded in the past year.”
• “100% of drugs in our doctors’ bags should be in-date.”
•  
• The standard should reflect the clinical and medico-legal
importance of the criterion.
• in the example above, 80% of women should have had a
cervical smear,
• But of those who've had an abnormal smear, 100%
should have had action taken.
 Clinical Audit

Methodology
• Plan methodology in detail.
• Keep it simple.
• Allow sufficient time.
• Share ownership, involve others.
• Get co-operation from those who might be effected.
• Carry your colleagues with you.
 Standards
• How to set standards
• Look at national guidelines – EHSTG, HSTQ
• Literature (journals), textbooks
• Local guidelines
• Discussion with consultants/GPs
• Discussion with trainer/partners
• KEY POINT : Standards set should be realistic
and attainable. Justifiable reasons for the
standard set should be made explicitly clear.
 Standards

• Some criteria are so important that they need 100%

standard.
• However, 100% standards are unusual – patients or
circumstances usually conspire against perfection and
the standard needs to reflect that.
• Your literature search should give you an idea of what
standards others have managed to reach.
• Your standard needs to follow on directly from your
criterion – for example,
• “Patients on thyroxine should have had TFTs done in the
last year; this should have happened in at least 90% of
patients”.
 TYING IT ALL TOGETHER
Examples of Standards & Criteria
Criteria Standards

All children under 2 years should 90% of registered patients under

be immunised against tetanus and the age of 2 years should have
polio been immunised against polio and
tetanus
All notes of those patients with an 95% of patients with an allergy to
allergy to penicillin should be penicillin should be clearly marked
marked
All patients in the surgery should 70% of patients in the surgery
wait no longer than 30 minutes should wait no longer than 30
before a consultation minutes before a consultation
 Preparation & Planning
• Must show evidence of teamwork –
otherwise you will fail
 Data Collection (1)
• You can collect information from:
• computer registers
• review of contents of medical records
• questionnaires – patients, staff or GPs
• data collection sheets
 Data Collection
• Be careful of data collection – choose a topic
which does not entail too much data collection to
the extent it becomes exhaustive with
subsequent loss of enthusiasm
• ? Sampling – random or systemic
• Only collect essential information
• Use computers, ?data collection forms
• Use other staff & delegate – don’t do all the work
yourself
• Set a deadline
 Clinical Audit

Sample selection
• Sample selection:
– Retrospective
– Concurrent
• Sample size: appropriate
• Multi-professional
• All data should be anonymous
 Presenting the Results
• Collect Results
• Analyse Results
• Summarise Results
• Present Results to the team
• Simple arithmetic calculations
• Use percentages
• Results of 2nd data collection presented in
the same way as the 1st
Discussion – Data Collection 1
Comparing Results to Standards
Criterion Standard Observed Result

All patients should be 70% 45%

seen within 15
minutes of their
appointment
timeMinimum
70%45%All diabetics 95% 90%
to have had HbA1C in
last 3 months
Minimum
90%Drug allergies to 100% 95%
be marked as “active
problem” on computer
Minimum
Discussion – Data Collection (1)
• KEY POINT (Discussion of Data Collection
1) : You need to explain why you think the
practice didn't meet the standard that was
set.
 Discussion – why standards not
met
• Think: What reasons are there for practices not meeting audit
standards?
•  
• For example : reasons have included:
• Practice reasons:
• ·         Results having been put down as free text on computer,
rather than coded;
• ·         Opportunistic rather than formal recall system in use;
• Doctor reasons:
• ·         Not all GPs were aware of the practice policy;
• ·         Not all partners agreed with the policy;
• Patient reasons:
• ·         Patients refusing to have tests done;
• ·         Patients on holiday when tests due.
 Implementing Changes
• The most challenging stage
• Audit can tell you whether changes are
needed, but it can’t tell you what methods
to use
 Implementing Change
• The changes to be implemented should be
a team discussion and decision (?a
practice meeting)
• What to do at the Practice Meeting:
• Emphasise what has been achieved.
• What are we proud of?
• What are we not so proud of?
• How can we correct any deficiencies?
 Implementing Change
• Changes must be practical!
• How are you actually going to make the changes?
• Simply saying “We’ve got to do better” won’t result in
change
• You need to think through in detail
• ·         what needs to be done
• ·         who’s going to do it
• ·         when
• ·         and how.
• If you get very low results, you may consider resetting
the standards to a more realistic level (but justify it)
 Closing the Loop
Repeating the cycle 
• Re-evaluate care to ensure that any
remedial action has been effective.
• Audit is a continuous cycle – if you didn’t
meet the standard and you’ve planned
changes, you’ll need to repeat the audit to
make sure the changes have happened.
 Conclusions from the Audit
•   Summary of main issues learned
•    KEYPOINTS:
• Comment on any improvements that have resulted.
• How well did your proposals for change work?
• If you again didn't reach the standard that you set, why
not?
• If you did, should you be aiming higher next time, or look
at something else
• Where should the practice go from here
 Repeat the cycle
Select or Re-select
Topic

Evaluate Effects Define criteria

Of Change Set standards

Implement Plan
Changes Methodology

Collect & Analyze

Data
 Clinical Audit

IMPLEMENT REPEAT
CHANGES PROCESS

OBSERVE PRACTICE RECOMMENDATIONS

SET STANDARD

COMPARE
WITH
STANDARD
 Clinical Audit

Present audit data – write report

• Written report
• Show how actual practice compares with the audit
measures and how the audit findings relate to the
objectives (if re-audit were actions implemented?)
• Use the following headings:
• Care Group; Audit title; Audit team; Date;
Objectives; Measures/standards; Data Source;
Sample (timeframe); Results; Conclusions;
Recommendations; Actions
 How To Fail
• No justification for choice of audit
• No justification for criteria/standard settings
• Not having explicit criteria/standards
• Setting unreasonable standards
• A general lack of evidence based literature or using material that is not peer
referenced
• Not explicitly displaying teamwork in the “method” – must give specific
examples
• Numerical errors re: data collection
• Presentation of data collection eg no graphs, no percentages (ie the reader
has to do the hardwork him/herself)
• Not giving much thought to “changes to be evaluated” and not being specific
enough. Not delegating specific changes to specific people/persons.
• Poor conclusions and what the process has taught you
• No inclusion for possible sources of bias
• References not properly quoted
 Checking Session Understanding
• DISCUSS THE FOLLOWING STATEMENTS
• An example of the Audit of process is audit of referrals to hospitals.
• Audit usually consumes an extensive amount of resources (of time,
money etc.).
• Rare conditions should be audited.
• The higher the standard the practitioner starts with, the stronger is the
resulting audit.
• Maintaining clearly written notes of at least 20% of patients who are
sensitive to penicillin is an acceptable standard in general
• practice.
• The higher the amount of data the practitioner collects, the easier is the
decision making process in audit.
• The most challenging stage in Audit is implementing change.
• In data collection all in the target population must be included.
• The agreed standards can be reset at realistic percentages after the first
round of data collection.