Patient and public involvement in national clinical effectiveness processes: a

systematic review
Veronica Lambert1,*, Anne Matthews1, Riyad El-Moslemany1, Olive O’Connor2
1
School of Nursing and Human Sciences, Faculty of Science and Health, Dublin City
University, Ireland
2
School of Psychology, NUI Galway, Ireland
*e-mail: veronica.lambert@dcu.ie

Abstract
Ensuring that clinical effectiveness processes (i.e. clinical practice guidelines and audit)
reflect the needs and concerns of patients may help translate recommendations into practice.
However, uncertainty exits on how best to integrate patient and public involvement (PPI) in
clinical effectiveness processes (CEPs). This review systematically reviewed available
evidence on PPI in the development and governance of national CEPs. Drawing on the Centre
for Reviews and Dissemination guidance for undertaking systematic reviews in healthcare and
the Preferred Reporting in Systematic Reviews and Meta-Analysis criteria we used a
comprehensive search methodology to retrieve published and unpublished evidence nationally
and internationally; including electronic databases, grey literature, clinical audit and clinical
guidelines organizations. Eligible documents had to refer to PPI in the development, and or
governance, of CEPs at a national, or equivalent, level. Data were extracted on PPI benefits,
barriers, enablers, approaches, supports and evaluation in CEPs. Narrative synthesis was
employed. From a total screening of 2,515 documents, 41 documents were identified as
eligible for inclusion in the review. Of these 41 documents, 13 were descriptive papers, 7
were primary research studies, 7 were toolkits/reference manuals, 6 were secondary reviews,
3 were evaluation studies, 2 were protocols, 2 were policy/strategy documents and 1 was a
research briefing. The review revealed evidence that PPI does take place in CEPs
internationally. However, robust empirical evidence on which PPI strategy or approach is
most effective was limited. Despite a lack of empirical evidence, the documents appraised in
this review do provide baseline data and valuable insights into the complex process of
integrating PPI into CEPs. Further research is needed to establish the effectiveness of
different PPI programs used in CEPs. Better evaluation of PPI approaches in CEPs could
potentially enhance the wider acceptance and development of PPIPs if seen to be effective.

Keywords: patient, public, involvement, clinical guidelines, clinical audit

1. Introduction
This presentation reports findings from a Department of Health (Ireland) commissioned
systematic review with the remit of identifying available evidence (published and
unpublished) to support, or not, public and patient (defined as patient or patient advocate)
engagement/involvement in the development and governance of national clinical effectiveness
processes (CEPs).

© The 2016 Healthcare systems Ergonomics and Patient Safety Conference (HEPS 2016) 249
2. State of the art
There is growing consensus about the crucial role of patient and/or patient representative,
public advocate involvement in CEPs, including clinical guideline development and audit
processes. This is important as health professionals perspectives on healthcare processes,
priorities and outcomes may differ from the perspectives and priorities of patients. Ensuring
that CEPs reflect the needs and concerns of patients may help with achieving the translation
of recommendations into clinical practice. However, difficulties can ensue in making patient
and public contribution effective as it remains unclear on how best to conduct this process of
lay stakeholder engagement in the context of clinical practice guidelines and clinical audit
processes. If the goal of involving patients and the public in CEPs is enhanced quality in
health care then there is a need to understand the design, processes and mechanisms of PPI in
CEP.

3. Objectives and Methods

The specific objectives of the review, as outlined in the tender document by the Irish
Department of Health who commissioned the review, were to;
1. Identify the available evidence on the benefits of patient engagement for clinical
practice generally, and, more specifically, in clinical effectiveness processes
2. Ascertain, from the evidence sourced, what barriers and enablers exist to patient
engagement for clinical practice generally and, more specifically, in clinical
effectiveness processes
3. Synthesis the evidence on the clinical effectiveness processes that patients are engaged
in; including
i. Summary of approaches used e.g. consultation, committee membership etc.
ii. Describe the reported benefits and weaknesses of each approach
4. Synthesis the evidence on the methods and systems, including training, that are in
place to engage and support patients in the development and governance of the CEPs
of clinical audit and clinical guidelines at national (or equivalent) level
5. Identify what measurement or evaluation has occurred in relation to patient
engagement or the systems and methods used to support patient engagement.
The review was conducted and reported in accordance with the Centre for Reviews and
Dissemination (2008) guidance for undertaking systematic reviews in healthcare and the
Preferred Reporting in Systematic Reviews and Meta-Analysis (PRISMA) criteria (Moher et
al. 2009), as far as possible. Alongside having an expert by experience as a key member of the
review team an Expert by Experience Advisory Group was established prior to
commencement of the review with the remit to offer expertise through lived experience,
patient advocate and service user involvement expertise on considering the emergent evidence
on PPI in CEPs.
A comprehensive search methodology was used to retrieve published and unpublished
evidence nationally and internationally; including electronic databases, grey literature, clinical
audit and guidelines organizations. We also scanned bibliographies of all included documents
and identified any other relevant information on unpublished and ongoing work by contacting
experts in the field. To be included eligible papers had to refer to PPI in the development, and
or governance, of national CEPs; inclusive of clinical audits and guidelines at a national, or
equivalent, level. No study design limits were applied and we included primary and secondary
research, descriptive pieces and reference manuals, toolkits, policies and strategies produced
by national, government and/or other relevant organizations with remit for clinical guideline
and audit processes.

© The 2016 Healthcare systems Ergonomics and Patient Safety Conference (HEPS 2016) 250
A two-stage screening process was undertaken were titles, abstracts and full text (as
appropriate) of potentially eligible documents were assessed independently by two reviewers
with any discrepancies resolved by discussion with a third reviewer. Duplicate data extraction
was also conducted by two independent reviewers with any discrepancies resolved by
consensus with a third reviewer. Initially data was extracted on the characteristics of the
document including bibliographic reference, aim, design/type of document and
sample/methods as appropriate. Following this, a table mapping each document to the review
objectives of benefits, barriers, enablers, approaches, support and evaluation was developed to
assist with data extraction from each document according to each of the review objectives.
To take account of diverse study design, we used three appraisal instruments to quality assess
primary and secondary research studies including; the critical appraisal skills programme
(CASP) tool for qualitative studies, a modified version of an appraisal tool designed by
Tsimicalis et al. (2005) for quantitative studies and AMSTAR (A Measurement Tool to
Assess Systematic Reviews) for secondary review papers. Owing to the heterogeneity of
evidence retrieved it was not possible to conduct a meta-analysis and/or a meta-synthesis,
therefore results were summarized narratively according to the review objectives.

4. Results and Discussion

From a total screening of 2,515 documents, we identified 41 documents as eligible for
inclusion in the review. Of these 41 documents 13 were discursive/descriptive/opinion pieces,
original 7 were primary research studies, 7 were toolkits/reference manuals, 6 were secondary
review papers, 3 were evaluation studies, 2 were protocols, 2 were policy/strategy documents
and 1 was a research briefing. The results of the quality assessment of the 13 primary and
secondary research studies found the majority of studies to be of moderately high quality,
despite some limitations of the appraisal tools used. Quality ratings are representative not only
of study quality but are also reflective of methodological reporting and the match between the
quality appraisal tool and the design of the research studies.
This review revealed evidence that PPI in national CEP does take place internationally.
However, robust empirical evidence on which PPI strategy or approach is most effective was
limited. The majority of documents reviewed reported on PPI in clinical guideline
development with a dearth of data on PPI integration in clinical audit processes. In
considering the review objectives, some of the main findings identified are summarized
below.

Benefits of PPI in CEPs

Despite a general consensus that patient and public representatives should be involved in
CEPs, the added benefits of PPI in CEFs has yet to be established empirically.
Notwithstanding this, the difficulty, or perhaps impossibility, of examining the effects of
patient participation using randomized controlled trials was acknowledged, in addition to, the
fact that decision-making processes may need to be studied in different ways.

Barriers and facilitators to PPI in CEPs

The review identified a number of potential barriers and facilitators to PPI in CEPs. As dual
barriers and facilitators to PPI, core issues to take account of included; the representation and
selection process for patient and public representatives; transparency in terms of the roles and
responsibilities of patient and public representatives; training and support mechanisms, use of
a range of PPI approaches, being committing to and valuing PPI and working in a mutually
respectful environment.

© The 2016 Healthcare systems Ergonomics and Patient Safety Conference (HEPS 2016) 251
Approaches to PPI in CEPs
The three main PPI strategies identified in this review were consultation, participation and
communication. While there was limited data available on evidence based outcomes on the
strengths and weaknesses of these three PPI strategies it was recognised that each strategy has
its own strengths and limitations. Consequently, it was acknowledged that effective
involvement should begin with finding the best approach tailored to the specific PPI goal in
any given context; and the level of involvement should be clear and transparent for all
concerned. Representation of lay members was often restricted to a select number of patient
or patient representatives/organisations and did not by large include a diverse population of
patients and/or the general public.

Methods and systems to support PPI in CEPs

The consensual evidence is that patient representatives should be trained, prepared, guided
and educated for their role. Practical, emotional and financial assistance, as appropriate,
should be provided. Limited reporting existed on the model, mode, delivery, timing, content,
trainers, cost, evaluation of and effective impact of various training and support mechanisms.

Evaluation of PPI approaches or systems to support PPI in CEP

There was a paucity of rigorous process and impact evaluations to determine the effectiveness
of PPI approaches, and/or methods and systems to support PPI, in CEPs.

5. Conclusion and perspectives

This review provides baseline data and valuable insights into the complex process of
integrating PPI into CEPs. Some key principles to consider for integrating PPI into national
CEPs are outlined below:
1. Consideration should be given to the integration of PPI into these processes to
strengthen public participation in healthcare decision-making and to bring expert
experiential knowledge to these processes.
2. The three PPI strategies of consultation, participation and communication can be
employed as required in each CEP, and full active public/patient participation should
be explored where appropriate.
3. The most appropriate patient and public representation should be examined for each
case, drawing on public, patient, carer and other peer or lay representatives; while
there is no evidence to recommend one approach to the selection and recruitment of
patient and public representatives though a transparent process is required.
4. There is a need for comprehensive support for patient and public representatives,
specifically in terms of support from the chair of the guideline development group,
training, remuneration/compensation, physical, psychosocial and emotional support.
5. Several international organisations (e.g. NICE in the UK, SIGN in the UK, G-I-N
International Network, HQIP in the UK) have developed structured PPI programmes,
with supporting resources, to underpin their clinical effectiveness approaches. These
offer potentially valuable models to examine further for any framework development.
6. There is a need for further research into the effectiveness of different approaches to
integrating PPI into clinical effectiveness processes. Better evaluation of PPI in
clinical effectiveness processes could potentially enhance the wider acceptance and
development of PPIP’s if seen to be effective.

© The 2016 Healthcare systems Ergonomics and Patient Safety Conference (HEPS 2016) 252
Acknowledgements
This review was funded by the Department of Health (DoH) and overseen by the DoH
Clinical Effectiveness Unit. We would like to acknowledge Ms Olive O’Connor, expert by
experience, for her unique and invaluable contribution to the research team and for
confirming to us the true value of taking seriously the wisdom that expertise by experience
brings to all research and practice. We were also guided by members of our expert advisory
group, Stephen McMahon, Director of the Irish Patients’ Association and Dr. Liam
MacGabhann, Lecturer in Mental Health and expert on service user/patient engagement. We
would also like to acknowledge the Irish National Clinical Effectiveness Committee for their
insights and feedback.

References
Centre for Reviews and Dissemination. (2008)/ Systematic Reviews: CRD’s guidance for
undertaking reviews in health care. University of York, Centre for Reviews and
Dissemination.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D., PRISMA Group. (2009). Preferred Reporting
Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS
Medicine, 6(6), e1000097.
Shea, B.J., Hamel, C., Wells, G.A., Bouter, L.M., Kristjansson, E., Grimshaw, J., Henry,
D.A., Boers, M. (2009). AMSTAR is a reliable and valid measurement tool to assess the
methodological quality of systematic reviews. Journal of Clinical Epidemiology, 62(10),
1013-20.
Tsimicalis, A., Stinson, J., Stevens, B. (2005). Quality of life of children following bone
marrow transplantation: critical review of the research literature. European Journal of
Oncology Nursing, 9, 218-38.

© The 2016 Healthcare systems Ergonomics and Patient Safety Conference (HEPS 2016) 253