Chapter 3: Defining the criteria for including studies and how they will be grouped for the

synthesis

1) Eligibility criteria of studies in a review are based on the PICO elements of the review
question, types of study design, publication status, and language of published studies.

2) The population, interventions and comparators in the review question usually translate
directly into eligibility criteria for the review. Outcomes usually are not part of the criteria for
including studies, and a Cochrane Review would typically seek all sufficiently rigorous
studies of a particular comparison of interventions in a particular population of participants,
irrespective of the outcomes measured or reported.

Defining  The types of participants included in studies in a review should be

Participants sufficiently broad to encompass the likely diversity of studies and scenarios
in which the interventions will be used, but sufficiently narrow to ensure
that a meaningful answer can be obtained when studies are considered.
 The selection of participants can be based on the following attributes:
Disease of  This includes the type and severity of a disease or
Interest condition.
 E.g. In a review of platelet-rich therapies for
musculoskeletal soft tissue injuries, a subgroup analysis
was undertaken to examine if the effects of platelet-rich
therapies were modified by the type of condition (e.g.
rotator cuff tear, anterior cruciate ligament
reconstruction, chronic Achilles tendinopathy).
 E.g. In a review of beta-blockers for heart failure,
subgroup analyses were specified to examine if the
effects of beta-blockers are modified by the severity of
heart failure (‘reduced left ventricular ejection fraction
(LVEF)’ ≤ 40%, ‘mid-range LVEF’ > 40% and < 50%,
‘preserved LVEF’ ≥ 50%, mixed, not specified).
Patient  This involves deciding whether a specific population
Characteristics group is within scope, determined by factors such as
of Interest age, sex, race, socioeconomic status, and educational
status.
 E.g. In a review of antidepressants for depressive
disorders in children and adolescents, a subgroup
analysis was undertaken to examine if the effects of the
antidepressants were modified by age. The age groups
were defined as ‘children’ (aged 6 to 12 years) and
‘adolescents’ (aged 13 to 18 years).
Setting of  Interest may also focus on a particular setting (e.g.
Interest primary care vs. tertiary hospital, urban vs. rural, low
and middle-income countries vs. high-income countries,
institutional care vs. community-dwelling, inpatient vs.
 outpatient).
 E.g. In a review of hip protectors for preventing hip
fractures in older people, separate comparisons were
specified based on setting (institutional care or
community-dwelling) for the critical outcome of hip
fracture.
Recipient of  Authors may want to determine the intended recipient
Interest of the intervention (e.g. patient, carer, healthcare
provider, health system, policy maker, community).
 E.g. In a review of e-learning programmes for health
professionals, a subgroup analysis was planned to
examine if the effects were modified by the type of
healthcare provider (doctors, nurses or
physiotherapists).

Defining  It is useful to consider exactly what intervention is delivered, who delivers

Intervention it, how it is delivered, where it is delivered, when and how much is
s delivered, and whether the intervention can be adapted or tailored.
 Clearly defined intervention groups serve two main purposes in the
synthesis. First, the way in which interventions are grouped for synthesis
(meta-analysis or other synthesis) is likely to influence review findings.
Second, the intervention groups specified in a protocol provide a
standardized terminology for describing the interventions throughout the
review, overcoming the varied descriptions used by study authors.
 A process for planning intervention groups for synthesis:
Identify  Consider whether differences in interventions
Intervention characteristics might modify the size of the intervention
Characteristics effect importantly. This should be informed by
that May literature and expertise.
Modify the  E.g. In a review of the effectiveness of exercise in
Effect of the osteoarthritis, different exercise interventions may offer
Intervention different mechanisms (e.g. muscle strengthening, gait
and balance, pain relief, etc). The different mechanisms
of exercise might suggest different ways of grouping
interventions (e.g. by intensity, mode of delivery)
according to potential modifiers of the intervention
effects.
Label and  For each intervention group, provide a short label and
Define describe the core characteristics that will be used to
Intervention assign each intervention from an included study to a
Groups group.
 Groups can be named by intervention content (e.g. a
specific drug or a behaviour change technique), the
purpose or theoretical underpinning, mode of delivery,
provider, dose or intensity, duration or timing of the
 intervention.
 E.g. In a review of psychological therapies for panic
disorder, eight types of therapy were specified, namely
psychoeducation, supportive psychotherapy,
physiological therapies, behaviour therapy, cognitive
therapy, cognitive behaviour therapy (CBT), third-wave
CBT, and psychodynamic therapies. These groups were
defined by the theoretical basis of each therapy (e.g.
CBT aims to modify maladaptive thoughts through
cognitive restructuring) and the component techniques
used.
 In reviews of effectiveness of exercise in stroke
rehabilitation, exercise intensity may be defined by
training time (session length, frequency, program
duration), amount of work (e.g. repetitions), and
effort/energy expenditure (exertion, heart rate).
Determine  In some fields, intervention taxonomies and frameworks
Whether there have been developed for labelling and describing
is an Existing interventions, and these can make it easier for those
System for using a review to interpret and apply findings.
Grouping  Authors should use an agreed and validated terminology
Interventions to develop intervention groupings.
 E.g. The behaviour change technique (BCT) taxonomy
categorizes intervention elements into goal setting, self-
monitoring and social support. A review of social media
interventions used this taxonomy to describe
interventions and examine different BCTs as potential
effect modifiers.
Plan How the  Decide whether it is useful to pool all interventions in a
Specified single meta-analysis (‘lumping’), within which specific
Groups will characteristics can be explored as effect modifiers (e.g.
be Used in in subgroups).
Synthesis and  Alternatively, if pooling all interventions is unlikely to
Reporting address a useful question, separate synthesis of specific
interventions may be more appropriate (‘splitting’).
 E.g. In a review of exercise for knee osteoarthritis, the
different categories of exercise were combined in a
single meta-analysis, addressing the question ‘what is
the effect of exercise on knee osteoarthritis?’ The
categories were also analysed as subgroups within the
meta-analysis to explore whether the effect size varied
by type of exercise.
Decide How  Some complex interventions include multiple
to Group components that could also be implemented
Interventions independently. These components might be eligible for
with Multiple inclusion in the review alone, or eligible only if used
 Components alongside an eligible intervention.
or Co-  Options for considering multi-component interventions
Interventions may include the following:
 Identifying intervention components for meta-
regression or a components-based network meta-
analysis.
 Grouping based on the ‘main’ intervention
component.
 Specifying a separate group (‘multi-component
interventions’). ‘Lumping’ multi-component
interventions together may provide information
about their effects in general; however, this
approach may lead to unexplained heterogeneity
and/or inability to identify which components are
effective.
 Reporting results study by study. An option if
components are expected to be so diverse that
synthesis will not be interpretable.
 Excluding multi-component interventions. An
option if the effect of the intervention of interest
cannot be discerned. This approach may reduce the
relevance of the review.

 In a review of psychological therapies for panic

disorder, two of the eight eligible therapies
(psychoeducation and supportive psychotherapy) could
be used alone or as part of a multi-component therapy.
When accompanied by another eligible therapy, the
intervention was categorized as the other therapy (i.e.
psychoeducation + cognitive behavioural therapy was
categorized as cognitive behavioural therapy).

Defining  Cochrane Reviews sometimes include one comparison, but most often
Comparison include multiple comparisons.
s  Three commonly identified types of comparisons include the following:
Intervention  These include placebo drug, sham surgical procedure,
vs. Placebo psychological placebo.
 This aims to establish the effectiveness of an
intervention.
Intervention  Control is defined as no intervention, wait-list control, or
vs. Control usual care. Unlike placebo, both intervention and control
may receive standard therapy.
 This aims to establish the effectiveness of an
intervention.
 E.g. Chemotherapy plus best supportive care (BSC)
 versus BSC alone for palliative treatment of
gastroesophageal-junction carcinoma.
Intervention  A comparison of active interventions may include
A vs. comparison of the same intervention delivered at
Intervention different time points, for different lengths of time or
B different doses, or two different interventions.
 This aims to compare the effectiveness of two
interventions.
 E.g. Early (commenced at less than 2 weeks of age)
versus late (2 weeks of age or more) parenteral zinc
supplementation in term and preterm infants.

 In reviews including multiple intervention groups, many comparisons are

possible. Some authors compare effectiveness of all their included
interventions, including where there may be only indirect comparison of
some interventions across the included studies. However, authors may also
select a limited subset of the possible comparisons.
Defining  Generally, studies outcomes should not be a criterion for study eligibility,
Outcomes except the following reasons:
 When the review addresses the potential for an intervention to prevent a
particular outcome
 When the review addresses a specific purpose of an intervention that
can be used in the same population for different purposes.
 When the outcomes are essential for decision or policy making.

 Outcomes may include survival, mortality, clinical events (e.g. strokes or

myocardial infarction), behavioural outcomes (e.g. changes in diet, use of
services), patient-reported outcomes (e.g. symptoms, quality of life),
adverse events, burdens (e.g. demands on caregivers, frequency of tests,
restrictions on lifestyle) and economic outcomes (e.g. cost).
 Various sources can be used to develop a list of relevant outcomes,
including input from consumers and advisory groups, clinical experiences
of the authors, and evidence from the literature (including qualitative
research).
 Outcomes that will be included should be specified at the protocol stage.
 A process for planning outcome groups for synthesis:
 For each outcome, provide a short label and describe in
Fully Specify sufficient detail to enable eligible outcomes from each
Outcome included study to be categorized.
Domains  Outcomes can be defined hierarchically, starting with
very broad groups to specific outcome measures.
 E.g. In a review of interventions for sexual health
promotion, three broad outcome domains were defined
(cognitions, behaviours, biological) based on a
conceptual model. Each domain comprised more specific
outcomes (e.g. condom use, seeking health services such
 as STI testing).
Determine  Agreed terminology may help decision makers interpret
Whether review findings.
there is an  E.g. In a review of combined diet and exercise for
Existing preventing gestational diabetes mellitus, a core outcome
System for set agreed by the Cochrane Pregnancy and Childbirth
Grouping group was used.
Outcomes
Define the  In reviews, time frames, and not specific time points, are
Outcome often specified to handle the likely diversity in timing of
Time Points outcome measurement across studies.
 E.g. In a review of antidepressants, a clinically
meaningful time frame of 6 to 12 months might be
specified for the critical outcome ‘depression’, since this
is the recommended treatment duration.
Specify the  Measurement tools or methods provide an appropriate
Measurement assessment of the outcome (e.g. including clinical
Tool or assessment, laboratory tests, objective measures, and
Measurement patient-reported outcome measures.
Method  The measurement tools must have adequate reliability
and validity and are developed using theory, empirical
evidence and consumer involvement.
 E.g. In a review of exercise for osteoarthritis, measures
of pain were selected based on relevance of the content
and properties of the measurement tool (i.e. evidence of
validity and reliability).
Specify how  Multiple outcomes within a study may arise from
Multiplicity multiple measurement within a domain (e.g. ‘anxiety’
of Outcomes and ‘depression’ in a ‘mental health’ domain), multiple
will be methods to measure the outcome (e.g. self-reported
Handled depression, clinician-rated depression), multiple tools to
measure the outcome (e.g. Hamilton Depression Rating
Scale, Beck Depression Inventory), as well as multiple
time points measured within a time frame.
 To deal with this, select only one outcome per study for a
particular comparison.

3) Types of study designs eligible to be included in reviews:

 Randomised Trials
 Randomised trials are the best study
design for evaluating the efficacy of
interventions.
Non-Randomised Trial
 Non-randomised studies can be included:
 If randomised trials are unable to
address the effects of the intervention
in specific populations or settings.
 In circumstances where interventions
cannot be randomised (e.g. policy
change introduced in a single or small
number of jurisdictions).

 Authors need to decide whether results

from randomised trials and non-
randomised studies should be included in
the same synthesis (for the purpose of
examining whether study design explains
heterogeneity among the intervention
effects), or whether the effects should be
synthesised in separate comparisons.

4) Publication status: Obtaining and including unpublished studies (including grey literature)
can reduce the effects of publication bias. However, the unpublished studies that can be
located may be an unrepresentative sample of all unpublished studies.

5) Language of published studies: Authors should consider studies in any language as there
might be language bias if they restrict eligible studies to those published in one specific
language (usually English).

6) It will be worthwhile to pilot test the eligibility criteria on a sample of reports (e.g. six to
eight articles, including ones that are thought to be definitely eligible, definitely not eligible
and doubtful). The pilot test can be used to refine and clarify the eligibility criteria, train the
author who will be applying them and ensure that the criteria can be applied consistently by
more than one person.

7) Any changes to eligibility criteria in a review must be justified. In particular, post-hoc

decisions about inclusion or exclusion of studies should keep faith with the objectives of the
review rather than with arbitrary rules. Following pre-specified eligibility criteria is a
fundamental attribute of a systematic review. However, unanticipated issues may arise.
Review authors should make sensible post-hoc decisions about exclusion of studies, and
these should be documented in the review, possibly accompanied by sensitivity analyses.
Changes to the protocol must not be made on the basis of the findings of the studies or the
synthesis, as this can introduce bias.