Clinical Audits - Define Goals: Different Ways To Gain Insight

CLINICAL AUDITS – DEFINE GOALS
DIFFERENT WAYS TO GAIN INSIGHT

Priv.Doz.Mag.Dr. Gerald Sendlhofer
17.11.2018
TYPES OF AUDITS
 ISO 19011:2017 – Guidelines for auditing management
systems
 From global to a single process
 What should be audited
 Requirements defined in one or more management system standards (ISO 9001,
EFQM, JCI, etc.)
 Policies and requirements specified by other parties (Pain, Polytrauma, etc.)
 Legal requirements (Tissue Safety, Good Manufactering Practice, etc.)
 One or more management system processes defined by the organization or
other parties (admission, diagnosis, treatment, discharge)
 Particular structure, process steps – compliance (hand hygiene, medication
process, surgical safety checklist use, etc.)
17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 2

TYPES OF AUDITS
Boarders of the organization
Process or product audit
Depth of audit
System-audit Process-audit
17.11.2018
Scope of audit 3
TYPES OF AUDITS
 1st party auditing
– Internal auditing
 2nd party auditing
– External provider auditing
– Other external interested party auditing
 3rd party auditing
– Certification and/or accreditation auditing
– Legal, regulatory and similar auditing
UNDERSTAND THE ORGANIZATION
 Information should be provided to the auditor
e.g.
– Context of the organization
 Relevant internal and external requirements
 Goals of the organization
 Risks and chances

AIM OF AN AUDIT
 The aim of clinical audit is to establish whether
you are doing what you should be doing
– by asking if you are following agreed guidelines
– applying best practice
– And then making changes where necessary

AIM
 Clinical audit is not just data collection
 Data collection can be used for monitoring and
reporting
 Clinical audit converts data into information
that can improve practice

AUDIT METHODS
Scope of involvement between the Place of the Auditor
auditor and the organization to be
audited On-Site Remote
Human Interaction - Interview technique. About interactive communication

- with the participation of the organization medium:
to be audited. - Conduct surveys
- Check documents involving the - Complete checklists and questionnaires
organization to be audited. - Document review with the participation of the
- Take samples audited organisation
- Self and external assessment
No human interaction - Observation of the work carried out - Perform document check
(snapshot audits) (e.g. data analysis).
- Perform site inspection. - Monitor the work done, taking into account
- Complete checklists. social and legal requirements.
- Sampling (e.g., products). - Analyze data.

PROCESS-AUDITS
 Where to start: at the starting point or from the end
Problem
Reason
Reason
o x o x
Problem
 Several units involved

 Knowledge of process is necessary
 (Un)clarity which standards are relevant
 Completeness, adequacy and effectiveness (efficiancy,
effectiveness)

AUDIT - METHODS

INTERNAL AUDIT
 1st party audit
– Audit Programme
– Audit Plan
– Audit
 System, Process, Product/Service
– Audit Report with findings and deadline for feedback
– Re-Audit if necessary
– No certification
 Appr. 100 internal audits per year
EXTERNAL AUDIT
 2nd party auditing
– e.g. to prove the implementation of risk management,
GMP, GCP by external experts
– No certification
 Appr. 70 per year
 3rd party auditing
– Certification e.g. ISO 9001:2015, JCI, etc.
– Tissue safety act, GMP, GCP
 Appr. 20 per year

EXAMPLES

WHAT IS YOUR QUALITY ISSUE?
 What is the issue/problem?
 Define internal requirements to tackle these
issues/problems
 Train internal requirements
 Assess depth of penetration of requirements
– By those who are confronted with these
requirements

REMOTE AUDIT
SELF-AND EXTERNAL ASSESSMENT
 Illegibility and Incompleteness of doctors„
prescription
– Create a checklist
– Can be used for self and internal assessment

PROCESS

SELF-ASSESSMENT
 570 (285 first and 285 second assessment) fever curves were
self-assessed
– Doctors and nurses were confident with the legibility of their
selected fever curves; 79.3% at the first and 88.8% at the second
assessment stated “without errors”
– Doctors and nurses were optimistic regarding dosing and unit
errors; 71.2% of the fever curves at the first assessment and
77.0% at the second assessment “without dosing errors”
– Doctors and nurses were confident regarding the correctness of
the fever curves with respect to frequency of administration;
54.0% at the first assessment and 65.0% at the second were self-
rated as containing no errors
– Self-assessment of doctors and nurses

EXTERNAL ASSESSMENT
 By experts
– Legibility: no errors: 50.5% and 66.0%
(remember: 79.3% and 88.8%)
– Dosing and unit errors; 51.9% and 59.9%
– Frequency of administration; 30.3% and 42.6%

SELF-VERSUS EXTERNAL ASSESSMENT
Number of fever curves
without errors with respect
to legibility, dosing/unit and
frequency of administration
according to the self- (S) and
external (E) assessments

REMOTE AUDIT

REMOTE AUDIT – QUANTITATIVE
APPROACH
Sendlhofer, G; Lumenta, DB; Leitgeb, K; Kober, B;
Jantscher, L; Schanbacher, M; Berghold, A;
Pregartner, G; Brunner, G; Tax, C; Kamolz, LP
The Gap between Individual Perception and
Compliance: A Qualitative Follow-Up Study of the
Surgical Safety Checklist Application.PLoS One.
2016; 11(2):e0149212-e0149212

ONSITE AUDIT: SNAPSHOT AUDIT –
QUALITATIVE APPROACH
 Audits were performed on 36 operating days (2015: n=19; 2016:
n=17), in which a total of 136 surgical interventions were
observed.
 Overall, the percentage of "very good compliance" improved from
2015 to 2016:
for the sign-in from 52.9% to 81.2%
for the team-time-out from 33.3% to 58.8% (p=0.181),
for the sign-out from 21.4% to 41.7% (p=0.401).
 Snapshot audits are a valuable tool in the qualitative analysis

 Give more insight than a mere completeness check of ticks in SSC
documents
SNAPSHOT AUDIT
Sendlhofer, G; Lumenta, DB; Pregartner, G; Leitgeb, K; Tiefenbacher, P; Gombotz, V; Richter, C; Kamolz, LP; Brunner, G
Reality check of using the surgical safety checklist: A qualitative study to observe application errors during snapshot audits.
PLoS One. 2018; 13(9): e0203544-e0203544

ONSITE AUDIT
COMPLIANCE MEASUREMENT
Training: >6.000 HCP were trained over 5 years
Indirect measurement:
• How much alcohol-based hand rub disinfectant
was consumed
Direct measurement:
• Hygiene experts strictly followed the guideline for
conducting compliance measurements issued by
the German clean hands campaign.
Each of the M5M for HH was at least observed for
20 times during one compliance measurement
and at least 150 M5M for HH had to be observed
in each ward.
Register: Nosocomial Infection surveillance

From 2013 to 2017, a total of 84 compliance measurements with 19,295

moments for HH took place in 49 wards (moment 1= 4,807 observations,
moment 2=2,604 observations, moment 3=3,198 observations, moment
4=5,022 observations and moment 5=3,664 observations).
Hoffmann, M; Sendlhofer, G; Pregartner, G; Gombotz, V; Tax, C; Zierler, R; Brunner, G
Interventions to increase hand hygiene compliance in a tertiary university hospital over a period of five years: an iterative process of information, training and feedback.
J Clin Nurs. 2018;

Hoffmann, M; Sendlhofer, G; Pregartner, G;
Gombotz, V; Tax, C; Zierler, R; Brunner, G
Interventions to increase hand hygiene compliance
in a tertiary university hospital over a period of five
years: an iterative process of information, training
and feedback.J Clin Nurs. 2018;

ONSITE AUDIT AND
QUALITY IMPROVEMENT PROJECT

TIMELINE OF STUDY PHASES
Kopanz, J;
Lichtenegger, KM;
Sendlhofer, G;
Semlitsch, B; Cuder,
G; Pak, A; Pieber, TR;
Tax, C; Brunner, G;
Plank, J
Limited Documentation
and Treatment Quality
of Glycemic Inpatient
Care in Relation to
Structural Deficits of
Heterogeneous
Insulin Charts at a
Large University
Hospital.
J Patient Saf. 2018;

FINDINGS
Kopanz, J; Lichtenegger, KM; Sendlhofer, G;

Semlitsch, B; Cuder, G; Pak, A; Pieber, TR;
Tax, C; Brunner, G; Plank, J
Limited Documentation and Treatment
Quality of Glycemic Inpatient Care in Relation
to Structural Deficits of Heterogeneous
Insulin Charts at a Large University Hospital.

ONSITE AUDIT AND
QUALITY IMPROVEMENT PROJECT
Kopanz, J; Lichtenegger, KM; Sendlhofer, G;

Semlitsch, B; Cuder, G; Pak, A; Pieber, TR;
Tax, C; Brunner, G; Plank, J
Limited Documentation and Treatment
Quality of Glycemic Inpatient Care in Relation
to Structural Deficits of Heterogeneous
Insulin Charts at a Large University Hospital.

PEER REVIEW
 Austrian Inpatient Quality Indicator (A-IQI)
– Quality Indicators from the Ministry of Health
 e.g. in-house mortality rates (stroke, heart failure, etc.)
– Green-Yellow-Red-Traffic light system
– Patient record chart review
– Peer Review of patient records
– Peer Review findings and leassons learned

REGISTER FOR BENCHMARKING
Mortality Rate of Trauma patients

WHICH AUDIT TYPE IS THE BEST?
 It depends on the maturity of the
organization/topic
 Start with small interventions for a topic
– Iterative process
 Identify topic/problem
 Analyse the problem
 Find solution
 Training and implementation
 Check (audit)
 Training
 Check (audit), …
 Combine approaches
– self- and external assessment
– compliance measurements/snapshot audits
– Internal and external audits
 Why?
– Performing a self-assessment gives doctors/nurses the opportunity
to evaluate their performance against objective criteria
– Through compliance measurements data are generated and HCW
have a learning curve
– Through audits internal or external criteria/guidelines are checked

Problem – Solution: Implementation of a topic
Training
Remote & On-site checks and 1st party audit

Self- and external assessment / Compliance measurement /
Snapshot audits, internal audits
2nd Party audit

3rd Party audit

TAKE HOME MESSAGE
 Clinical audit is not a tool to criticise practice,
but to improve a system
 Which type of audit you use, is dependent on
the majurity of the system
– Most mistakes are due to limitations
– Failures within a system
© anaQuestra.com
MUSTER
EvaSys Self-assessment and external evaluation_2018
LKH-Univ. Klinikum Graz
Bitte so markieren: Bitte verwenden Sie einen Kugelschreiber oder nicht zu starken Filzstift. Dieser Fragebogen wird maschinell erfasst.
Korrektur: Bitte beachten Sie im Interesse einer optimalen Datenerfassung die links gegebenen Hinweise beim Ausfüllen.
1. General information
1.1 Date of Self-assessment
/ /
1.2 ID-number case history
/
1.3 Department of...
1.4 Division of...
1.5 Ward...
1.6 Name of examiner (doctor)
1.7 Are you a Risk manager? yes no
1.8 Name of examiner (nurse)
1.9 Are you a Risk manager? yes no
1.10 Name of examiner (other)
F2520U0P1PL0V0 26.04.2018, Seite 1/4
MUSTER
MUSTER
2. Self-assessment
To
so
m
e
Pa
no
ex
rti
te
ta
N
al
nt
ot
ly
pp
Fu
fu
fu
fu
lic
lfi
lfi
lfi
lfi
ab
lle
lle
lle
lle
le
d
d
2.1 Are prescription legible written on the
fever chart?
Fulfilled: all prescriptions are legible
Partially fulfilled: one prescriptions is illegible
To some extent fulfilled: two prescriptions are illegible
Not fulfilled: more than two prescriptions are illegible
2.2 Are prescription written with a

waterproofed pen?
Fulfilled: all prescriptions are written with a waterproofed pen
Partially fulfilled: one prescriptions is not written with a waterproofed pen
To some extent fulfilled: two prescriptions are not written with a waterproofed pen
Not fulfilled: more than two prescriptions are not written with a waterproofed pen
2.3 Is the available generic medication

prescribed on the fever chart?
Fulfilled: all generic medications are prescribed
Partially fulfilled: one generic medication is not prescribed
To some extent fulfilled: two generic medications are not prescribed
Not fulfilled: more than two generic medications are not prescribed
2.4 Are medications prescribed with the dosing

including the unit? (true: 15mg; false: 1 vial)
Fulfilled: all medications include dosage and unit
Partially fulfilled: one medication includes no dosage and unit
To some extent fulfilled: two medications do not include dosage and unit
Not fulfilled: more than two medications do not include dosage and unit
2.5 Are medications prescribed including the

concentration? (true: 500mg in 100ml
NaCl 0,9%; false: ad short infusion)
Fulfilled: all medications include concentration
Partially fulfilled: one medication includes no concentration
To some extent fulfilled: two medications do not include the concentration
Not fulfilled: more than two medications do not include the concentration
To
so
m
e
Pa
N
ex
rti
ot
te
N
al
ap
nt
ot
ly
Fu
pl
fu
fu
fu
ic
lfi
lfi
lfi
lfi
ab
lle
lle
lle
lle
le
d
2.6 Is the time interval of administration

prescribed correctly? (true: 1-0-1-0;
false: 2x1)
Fulfilled: all medications include time time intervals
Partially fulfilled: one medication includes no time intervals
To some extent fulfilled: two medications includes no time intervals
Not fulfilled: more than two medications includes no time intervals
F2520U0P2PL0V0 26.04.2018, Seite 2/4
MUSTER
MUSTER
2. Self-assessment [Fortsetzung]
2.7 Are medications written out? (true:

Neodolpasse; false: NDP)
Fulfilled: all medications are written out
Partially fulfilled: one medication is not written out
To some extent fulfilled: two medications are not written out
Not fulfilled: more than two medications are not written out
2.8 Are prescriptions signed by doctors?

Fulfilled: all medications are written out
Partially fulfilled: one medication is not written out
To some extent fulfilled: two medications are not written out
Not fulfilled: more than two medications are not written out
2.9 Are changes of a prescription signed by

doctors?
Fulfilled: all changes of prescriptions are signed
Partially fulfilled: one prescriptions is not signed
To some extent fulfilled: two prescriptions are not signed
Not fulfilled: more than two prescriptions are not signed
2.10 Are prescriptions crossed out in case of

changes so that they remain legible?
Fulfilled: all changes of prescriptions are recognizable crossed out
Partially fulfilled: one change of prescription is not recognizable crossed out
To some extent fulfilled: two changes of prescription are not recognizable crossed out
Not fulfilled: more than two changes of prescription are not recognizable crossed out
2.11 Are medications only prescribed in the

right column?
Fulfilled: all medications are prescribed in the right column
Partially fulfilled: one medications is not prescribed in the right column
To some extent fulfilled: two medications are not prescribed in the right column
Not fulfilled: more than two medications are not prescribed in the right column
2.12 In one line there is only one medication

prescribed?
Fulfilled: each medication is prescribed in one line
Partially fulfilled: in one line two medications are prescribed
To some extent fulfilled: in two lines two or more medications are prescribed
Not fulfilled: in more than two lines two or more medications are prescribed
To
so
m
e
Pa
N
ex
rti
ot
te
N
al
ap
nt
ot
ly
Fu
pl
fu
fu
fu
ic
lfi
lfi
lfi
lfi
ab
lle
lle
lle
lle
le
d
2.13 Are medications prescribed for the next day?

Fulfilled: all medications are prescribed for the next day
Partially fulfilled: one medications is not prescribed for the next day
To some extent fulfilled: two medications are not prescribed for the next day
Not fulfilled: more than two medications are not prescribed for the next day
2.14 For pro re nata medication the reason is

defined (e.g. upon pain, upon vomiting)?
F2520U0P3PL0V0 26.04.2018, Seite 3/4
MUSTER
MUSTER
2. Self-assessment [Fortsetzung]
Fulfilled: for all medications the reason is defined
Partially fulfilled: for one medication the reason is not defined
To some extent fulfilled: for two medications the reason is not defined
Not fulfilled: for more than two medications the reason is not defined
2.15 For pro re nata medication the daily

maximum dose is defined (e.g. up to 3
times per day, every 8 hours)?
Fulfilled: for all medications the maximum dose is defined

Partially fulfilled: for one medication maximum dose is not defined
To some extent fulfilled: for two medications maximum dose is not defined
Not fulfilled: for more than two medications maximum dose is not defined
2.16 Remarks:
Thank you for your cooperation.
F2520U0P4PL0V0 26.04.2018, Seite 4/4
MUSTER

Clinical Audits - Define Goals: Different Ways To Gain Insight

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Clinical Audits - Define Goals: Different Ways To Gain Insight

Uploaded by

Copyright:

Available Formats

CLINICAL AUDITS – DEFINE GOALS

DIFFERENT WAYS TO GAIN INSIGHT

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 2

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 5

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 6

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 7

Human Interaction - Interview technique. About interactive communication

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 8

 Several units involved

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 9

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 10

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 12

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 13

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 14

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 15

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 16

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 17

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 18

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 19

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 20

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 21

 Snapshot audits are a valuable tool in the qualitative analysis

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 23

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 24

From 2013 to 2017, a total of 84 compliance measurements with 19,295

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 25

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 26

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 27

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 28

Kopanz, J; Lichtenegger, KM; Sendlhofer, G;

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 29

Kopanz, J; Lichtenegger, KM; Sendlhofer, G;

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 30

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 31

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 32

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 34

Remote & On-site checks and 1st party audit

2nd Party audit

17.11.2018 Priv.Doz.Mag.Dr. Gerald Sendlhofer 35

1.4 Division of...

1.6 Name of examiner (doctor)

1.7 Are you a Risk manager? yes no

1.8 Name of examiner (nurse)

1.9 Are you a Risk manager? yes no

1.10 Name of examiner (other)

F2520U0P1PL0V0 26.04.2018, Seite 1/4

2.2 Are prescription written with a

2.3 Is the available generic medication

2.4 Are medications prescribed with the dosing

2.5 Are medications prescribed including the

2.6 Is the time interval of administration

F2520U0P2PL0V0 26.04.2018, Seite 2/4

2.7 Are medications written out? (true:

2.8 Are prescriptions signed by doctors?

2.9 Are changes of a prescription signed by

2.10 Are prescriptions crossed out in case of

2.11 Are medications only prescribed in the

2.12 In one line there is only one medication

2.13 Are medications prescribed for the next day?

2.14 For pro re nata medication the reason is

2.15 For pro re nata medication the daily

Fulfilled: for all medications the maximum dose is defined

Thank you for your cooperation.