Professional Documents
Culture Documents
Writing A Systematic Review: An Introduction: Hening Pujasari, PHD
Writing A Systematic Review: An Introduction: Hening Pujasari, PHD
AN INTRODUCTION
Presented at
One Day Training Scientific Writing: Systematic Review
FoNUI Post Graduate Student Association
November 6, 2021
11/09/2021 pujasari@ui.ac.id
Outline what a systematic review is
REVIEW? REVIEW: A
critical appraisal of a
book, play or other
work
11/09/2021 pujasari@ui.ac.id
What is a systematic review?
“A systematic review is a review in which there is a comprehensive search for relevant
studies on a specific topic, and those identified are then appraised and synthesized
according to a predetermined and explicit method.”* (*Klassen et al. Guides for reading
and interpreting systematic reviews. Arch Pediatr Adolesc Med 1998;152:700-704.)
A systematic review attempts to collate all empirical evidence that fits pre-specified
eligibility criteria in order to answer a specific research question. It uses explicit,
systematic methods that are selected with a view to minimizing bias, thus providing
more reliable findings from which conclusions can be drawn and decisions made
(Antman 1992, Oxman 1993)
11/09/2021 pujasari@ui.ac.id
Minimise the impact of
bias/errors
Can help to end confusion
Highlight where there is not
Why we need sufficient evidence
systematic Combining findings from
reviews different studies can highlight
new findings
Can mitigate the need for
further trials
11/09/2021 pujasari@ui.ac.id
Why we need systematic reviews
• Facilitate rational decision making
• Health care providers, researchers and policy makers are
inundated with unmanageable amounts of information
• Over 20 million citations in PubMed
• Approx. 75 to 100 RCTs published daily
• Usually impossible to consider all relevant individual
primary research studies in a decision-making context
• Enable practitioners to keep up to date with evidence
accumulating in field and to practice evidence-based
medicine
11/09/2021 pujasari@ui.ac.id
‘Unscientific’ rarely pre-specify or make
methods explicit
11/09/2021 pujasari@ui.ac.id
Hierarchy
of
evidence
11/09/2021 pujasari@ui.ac.id
Who undertakes systematic
reviews?
• Cochrane/Campbell Collaboration
• NICE/Regulatory bodies
• Health Technology Assessment
• Academics/researchers/Clinicians
• Students
11/09/2021 pujasari@ui.ac.id
Multidisciplinary teams
Clinicians
Health Economists
11/09/2021 pujasari@ui.ac.id
Key Stage Systematic Review- the process
Data extraction /checking
Define research/review question
Develop data extraction from into which study
In consultation/collaboration with the information and outcome data can be extracted,
clinical community, commissioners and checked & verified
patient/public representatives
Study assessment/appraisal
Develop review protocol
Assess the quality and validity of the included
Pre-specify the type of studies to be studies using the pre-defined method.
included, the methods of collating,
appraising and analysing data
Synthesis
Identify relevant studies
Narratively and/or statistically
Develop a comprehensive search summarise/describe the data, exploring
strategy and undertake systematic similarities and differences between studies.
searches of the literature
Assess eligibility
Knowledge translation
Select those studies which meet the pre-
defined inclusion criteria Review details and results are disseminated to
relevant target audiences using appropriate
formats
11/09/2021 pujasari@ui.ac.id
Questions may be broad or
narrow
Well-formulated questions will
guide many aspects of the
Define review process
research/revie Searching strategy
w question Inclusion/exclusion criteria
Data extraction
Choice of synthesis method
Presentation/dissemination of
findings
11/09/2021 pujasari@ui.ac.id
•Discuss a very broad question and how you might
narrow it? (10 mins)
Quick Activity
11/09/2021 pujasari@ui.ac.id
a clear and concise statement of a
review's objectives (or questions) is
critical and should begin with a
precise statement of the primary
objective, including the
interventions reviewed and the
targeted problem; ideally, this would
be presented in a single sentence
Current
Cochrane & Prisma Statement guidance
11/09/2021 pujasari@ui.ac.id
“To assess the effects of [intervention
or comparison] for [health problem] in
[types of people, disease or problem, and
setting if specified].” Current
guidance
Several criteria/frameworks proposed
to help guide question development
11/09/2021 pujasari@ui.ac.id
Question formulation
• Determining the scope is a decision dependent upon multiple
factors:
• Perspectives regarding a question’s relevance and potential
impact;
• Supporting theoretical, biologic and epidemiological
information;
• The potential generalizability and validity of answers to the
questions;
• Available resources;
• The wider literature base – has a recent high-quality SR
been conducted?
11/09/2021 pujasari@ui.ac.id
Question formulation
• Advantages and disadvantages to both broad and narrow
questions
11/09/2021 pujasari@ui.ac.id
Often dealing with complex interventions
Might be a need to develop working
definitions of the intervention of interest
Several options on how to do this (pragmatic
real world v theoretical, logic models, etc.)
Use content experts outside the review team
to ensure that the resulting definitions are
likely to be robust and meaningful
Question formulation
11/09/2021 pujasari@ui.ac.id
• A protocol is an essential component
of the systematic review process
• Helps to ensure careful a priori
planning
Protocol • Consistency
Development • Transparency
• Integrity
• Integral part of the process for
leading organizations/publication
process
11/09/2021 pujasari@ui.ac.id
One of the features that
distinguish a systematic review
from a narrative review is the
pre-specification of criteria
Protocol Inclusion
Development Exclusion
Methods
Outcomes to be synthesized
Etc.
11/09/2021 pujasari@ui.ac.id
PROSPERO – CRD initiative
• Search for existing current
reviews
• Register their planned review
online
• Publish protocol online
• Update record on Prospero
website as the review
progresses
• Avoids duplication of reviews
11/09/2021 pujasari@ui.ac.id
Types of
Studies (RCTs, non-RCTs, cohort/case-controlled)
Population and setting
Interventions
Outcome measures
www.york.ac.uk/inst/crd/pdf/Systematic_Reviews.pdf
http://handbook.cochrane.org/
11/09/2021 pujasari@ui.ac.id
Searching for Information
11/09/2021 pujasari@ui.ac.id
Word variants
• AIDS
• acquired immunodeficiency syndrome
• acquired immuno-deficiency syndrome
11/09/2021 pujasari@ui.ac.id
Searching for Information
Where to search
Electronic databases: Medline, Embase, Cochrane, PsycInfo, etc.
Grey literature, dissertations, theses, conference proceedings,
national bodies (NICE, HTA), clinical trial database (
www.clincialtrails.gov/)
11/09/2021 pujasari@ui.ac.id
Boolean
operators
11/09/2021 pujasari@ui.ac.id
Reference manager software package
Endnote – RefMan – ProCite – Mendeley
11/09/2021 pujasari@ui.ac.id
Data Extraction/Quality
Appraisal
11/09/2021 pujasari@ui.ac.id
Be clear what information you want from the studies:
• Study details
• Data for your analysis
11/09/2021 pujasari@ui.ac.id
What effect How are you
measures you planning to
are you going to group studies for
calculate the analysis?
• What data do you • By intervention?
need to do this? • By study design?
Give
consideration
What
information do
you need to
REMEMBER
YOUR to….
PROTOCOL – IT
extract to enable
IS YOUR
you to organise
ROADMAP,
and analyse the
FOLLOW IT!
way you want?
11/09/2021 pujasari@ui.ac.id
Level of judgement
However
is required
• Sufficient to • You need to limit
describe studies unnecessary
• Sufficient to allow detail
How much to you to undertake
the planned
extract?? analysis
• Sufficient so you
do not need to
return to the full
text papers
11/09/2021 pujasari@ui.ac.id
There is a wide selection of software to choose
from
11/09/2021 pujasari@ui.ac.id
Word
Excel
Access
EPPI reviewer
COEVIDENCE
REVMAN
????
Which
software?
11/09/2021 pujasari@ui.ac.id
Consistency/ • We all have to be doing the same thing
• Essential >one reviewer is extracting data
Standardization
• Data must be interpreted in the same way
by all reviewers
• Regular discussion of
progress/disagreements
• Regular comparison of data extraction –
don’t wait till the end
11/09/2021 pujasari@ui.ac.id
Once data extraction is complete you may need to:
11/09/2021 pujasari@ui.ac.id
• You may need separate forms for each study
Are you including design
more than one • However, you are still answering the same
question, so make sure the core information
study design? extracted is the same
Have one or a
• Will the data still be useful?
Things to few studies
• Should you include it?
reported data • Make sure the core information extracted is the
consider differently from same
the others?
11/09/2021 pujasari@ui.ac.id
Be careful about collecting ‘extra’ data
It is very tempting to collect data that are not
directly relevant to the review question
Why bother????
11/09/2021 pujasari@ui.ac.id
Qualitative studies
Three broad categories
Rigour: has a thorough and
Quality appropriate approach been
Assessment & applied to key research methods
in the study?
Critical Credibility: are the findings well
Appraisal presented and meaningful?
Relevance: how useful are the
findings to you and your
organisation?
11/09/2021 pujasari@ui.ac.id
Clear aims of research (goals, why it is
important, relevance)
Appropriate methodology
Sampling strategy
Data collection
CASP appraisal Relationship between researcher and
checklist participants
Ethical issues
Data analysis
8. Findings
9. Value of research (context dependent)
11/09/2021 pujasari@ui.ac.id
Data Synthesis
11/09/2021 pujasari@ui.ac.id
Results from different studies need to be synthesised
Are studies and results similar enough to be
combined into a single numerical result?
NO – qualitative descriptive/narrative summary
YES – quantitative meta-analysis
Data Synthesis Heterogeneity
Difference in results can arise due to differences in
study design, population, selection, intervention
delivery
How similar is similar? Results from heterogeneous
studies should not be pooled
11/09/2021 pujasari@ui.ac.id
Instead of/alongside meta-
analysis
Potential bias in presentation
Lack of a take home message
Narrative synthesis
11/09/2021 pujasari@ui.ac.id
Tools for Partly informed by methodological
work in qualitative synthesis
narrative
Tabulation
Groupings and clusters
synthesis
Vote counting as a descriptive tool
Examination of moderator variables
(elements of e.g. setting, population)
Rodgers et al Evaluation
2009 15 49-72
11/09/2021 pujasari@ui.ac.id
Meta-analysis/forest Plot
11/09/2021 pujasari@ui.ac.id
Most important thing:
Be organised!!!
11/09/2021 pujasari@ui.ac.id
Timeline
11/09/2021 pujasari@ui.ac.id
Use Use a reference manger to sift and store
11/09/2021 pujasari@ui.ac.id
Dates – YYYYMMDD
Version numbering
• v0.1 = first draft
• v1.0 = final version
• v1.1 = minor amendments to final version
Version control • v2.0 = major revision
11/09/2021 pujasari@ui.ac.id
DATA
EXTRACTION,
PRESENTING
FINDINGS, &
WRITING REPORT
11/09/2021 pujasari@ui.ac.id
DATA
EXTRACTION
11/09/2021 pujasari@ui.ac.id
Systematic Review Process Overview
11/09/2021 pujasari@ui.ac.id
To describe why data extraction is
important
To summarize studies in a
To obtain information to
common format to
To identify numerical data assess more objectively
facilitate synthesis and
for meta-analyses the risk of bias in and
coherent presentation of
applicability of studies
data
To identify systematically
missing or incorrectly
assessed data, outcomes
that are never studied, and
underrepresented
populations
11/09/2021 pujasari@ui.ac.id
On Data Extraction
• Is labor intensive
• Can be costly and error prone
11/09/2021 pujasari@ui.ac.id
Data Elements:
Population, Intervention, and Comparator
11/09/2021 pujasari@ui.ac.id
Data Elements: Timing and Study Design
11/09/2021 pujasari@ui.ac.id
Provide “operational definitions” (instructions)
Provide indicating exactly what should be extracted in
each field of the form.
Always
Provide
Instructions Make sure that all data extractors understand the
operational definitions the same way.
Make • Pilot-test the forms on several published papers.
• Encourage communication to clarify even apparently
mundane questions.
11/09/2021 pujasari@ui.ac.id
Single Versus Double Extraction
■ Independent extraction of data by at least two experienced reviewers is ideal but
is also resource intensive.
Forms Balance the structure of the form with the flexibility of its use.
(Evidence
Tables) Anticipate the need to capture unanticipated data.
Use an iterative process and have several individuals test the form on
multiple studies.
11/09/2021 pujasari@ui.ac.id
Common Problems Encountered When
Creating
Data Extraction Forms (Evidence Tables) (I)
11/09/2021 pujasari@ui.ac.id
Evidence Tables: Example (I)
Second Draft
First Draft
11/09/2021 pujasari@ui.ac.id
Evidence Tables: Example (II)
Final Draft
Common Problems Encountered When Creating
Data Extraction Forms (Evidence Tables) (II)
11/09/2021 pujasari@ui.ac.id
Sample Fields From a Table Guidance Document:
Vanderbilt University Evidence-based Practice Center
11/09/2021 pujasari@ui.ac.id
Example:
Two Reviewers Extract Different Data
Reviewer A Reviewer B
11/09/2021 pujasari@ui.ac.id
Samples of Final Data Extraction Forms
(Evidence Tables)
11/09/2021 pujasari@ui.ac.id
Pencil and paper
11/09/2021 pujasari@ui.ac.id
Extracting the Data
Berlin J, for the University of Pennsylvania Meta-analysis Blinding Study Group. Lancet 1997;350:185-6.
11/09/2021 pujasari@ui.ac.id
Problems in data
reporting
Challenges in
Inconsistencies in
Data
Extraction published papers
11/09/2021 pujasari@ui.ac.id
Data extraction is laborious and tedious.
Conclusions
Interpretation and subjectivity are unavoidable.
11/09/2021 pujasari@ui.ac.id
Key Messages
11/09/2021 pujasari@ui.ac.id
PRESENTATION OF
FINDINGS
11/09/2021 pujasari@ui.ac.id
Systematic Review Process Overview
Learning Objectives
11/09/2021 pujasari@ui.ac.id
Organizing and Reporting Findings
11/09/2021 pujasari@ui.ac.id
Why Not Meta-analysis?
11/09/2021 pujasari@ui.ac.id
Solution
Nonquantitative
synthesis using Use of evidence
tables that maps to provide an
summarize data overview of the data
across studies
11/09/2021 pujasari@ui.ac.id
PICOTS
Outcome: a hierarchy of
outcomes may reflect higher Setting: an intervention may
to lower acuity or target Timing: short term generally be available in various
outcomes followed by precedes long term settings (e.g., inpatient or
collateral ones; harms are outpatient)
generally presented last
11/09/2021 pujasari@ui.ac.id
Evidence tables are the first step to summarization, but
each evidence table represents the data in only one
study.
11/09/2021 pujasari@ui.ac.id
Combine data from multiple studies to
illustrate trends in the data
11/09/2021 pujasari@ui.ac.id
Summary Tables (II)
■ Simplified entry (one row) for each study •PICOTS = population, intervention,
■ Table columns may include, for example: comparator, outcomes, timing, and setting
– PICOTS (may be listed in table title or headers)
– Methodological quality
– Applicability
– Study size (weight)
– Magnitude of effect
■ A single study may be represented in multiple
summary tables (e.g., different outcomes)
11/09/2021 pujasari@ui.ac.id
Example:
Summary Table of Study Characteristics
11/09/2021 pujasari@ui.ac.id
Example:
Summary Table of Prevalence Findings
Summary tables
can be specialized
for different types
of questions.
Key
formats, respectively
Properly constructed summary tables:
Messages
Effectively convey results
Provide an overview of the literature in a
given field
Enable the reader to grasp results for
subsets of the literature
11/09/2021 pujasari@ui.ac.id
REPORTING THE REVIEW
11/09/2021 pujasari@ui.ac.id
Learning
Objectives
To distinguish examples of reporting that are adequate from those that are
inadequate
11/09/2021 pujasari@ui.ac.id
Systematic Review Process Overview
Writing the Report
PICOT(S) = population, intervention, comparator, outcome, time frame, and study design or setting
11/09/2021 pujasari@ui.ac.id
Abstract and Executive Summary
Chapter 1: Introduction
• The purpose of this chapter is to define the project, the
Systematic purpose and scope of the review, the key research questions,
the analytic framework, et cetera.
11/09/2021 pujasari@ui.ac.id
Systematic Review Report Structure (II)
Discussion
down by key questions) of the review, areas of future
research, and any conclusions that can be drawn.
11/09/2021 pujasari@ui.ac.id
Using Formal Guidelines To Improve
the Reporting of Systematic Reviews
The PRISMA (Preferred Reporting Items for Systematic
Reviews and Meta-Analyses) Statement is a guideline that
was developed to help improve the quality of review reports.
The guideline consists of a 27-item checklist and a flow
diagram.
Investigators can access the guideline online
(http://www.prisma-statement.org/index.htm).
11/09/2021 pujasari@ui.ac.id
Title of the Review Report
■ Identify the report as a systematic review, meta-analysis, or
comparative effectiveness review.
■ Use the PICOTS framework to guide construction of the title.
■ Make the title as succinct as possible yet keep it informative.
– Example of a short title:
Comparative Effectiveness of Lipid-Modifying Agents
– Example of a longer but more informative title:
Mortality in Randomized Trials of Antioxidant Supplements for
Primary And Secondary Prevention: Systematic Review and
Meta-analysis
PICOT(S) = population, intervention, comparator, outcome, time frame, and study design or setting
11/09/2021 pujasari@ui.ac.id
Executive Summary (I)
■ Should be structured as follows:
Background
Objectives
Key questions
Methods
Data sources
Eligibility criteria
Study appraisal
Data synthesis
Results
Limitations
Conclusions
Implications of key findings
11/09/2021 pujasari@ui.ac.id
Executive Summary (II)
Should be a distillation of the entire report
Should exclude study-by-study results
Should describe the evidence that supports all summary statements
Should describe the strength of the evidence as categorized in the evidence
review
11/09/2021 pujasari@ui.ac.id
Introduction
11/09/2021 pujasari@ui.ac.id
Methods Overview
Should provide the following information in a clear and transparent
manner:
Literature Search Strategy and Data Sources
Eligibility Criteria
Specific to study characteristics (e.g., PICOTS, length of followup)
Specific to report characteristics (e.g., years considered, language,
publication status)
Data Extraction and Data Items (e.g., variables for which data were
sought, assumptions and simplifications )
Quality Assessment
Synthesis of Results
Grading Strength of Evidence
Additional Analyses
11/09/2021 pujasari@ui.ac.id
Methods: Literature Search Strategy
Present the complete electronic search strategy — including any limits
used — in the Appendix of the report.
Brief example:
We used the following search terms to search all trials registers and
databases: immunoglobulin; IVIG [intravenous immunoglobulin];
sepsis; septic shock; septicaemia; and septicemia.
The purpose of including the entire search strategy is to ensure
transparency and to permit replication of the review.
11/09/2021 pujasari@ui.ac.id
Methods: Data Sources
Describe all information sources used in the literature search:
Databases with dates of coverage
Contacts with study authors to identify additional studies
Date of the last search
Example:
We searched the following databases for primary studies for the
periods in parentheses: MEDLINE® (1966 to January 2006),
EMBASE® (1974 to January 2006), and the Cochrane Central
Register of Controlled Trials (1966 to January 2006). We also
searched for systematic reviews until November 2005.
11/09/2021 pujasari@ui.ac.id
Methods: Eligibility Criteria (l)
State the processes used to select studies for review:
Screening
Eligibility assessment
Inclusion/exclusion criteria applied for the systematic review and, if applicable,
the meta-analysis
Example:
We included trials if the randomization scheme included groups that assigned
patients to treatment guided by the PAC [pulmonary artery catheter] or treatment
without the PAC. We only included trials if they reported death and number of
days hospitalized or the number of days in the ICU as outcome measures. Studies
were excluded if the randomization scheme did not specify groups as PAC or no
PAC, if patients were not randomized to a conventional PAC, if investigators
combined randomized and nonrandomized groups when reporting outcomes, or if
there were no outcome data on death or hospitalizations.
11/09/2021 pujasari@ui.ac.id
Methods: Eligibility Criteria (II)
Type of studies:
Example:
Randomised clinical trials studying the administration of hepatitis B
vaccine to CRF [chronic renal failure] patients, with or without
dialysis. No language, publication date, or publication status
restrictions were imposed.
Types of participants:
Example:
Participants of any age with CRF or receiving dialysis
(haemodialysis or peritoneal dialysis) were considered.…Renal
transplant patients were excluded from this review.
11/09/2021 pujasari@ui.ac.id
Methods: Eligibility Criteria (III)
Types of interventions:
Example:
Trials comparing the beneficial and harmful effects of hepatitis B vaccines with
adjuvant or cytokine co-interventions [and] trials comparing the beneficial and
harmful effects of immunoglobulin prophylaxis.…Hepatitis B vaccines (plasma or
recombinant [yeast] derived) of all types, dose, and regimens versus placebo,
control vaccine, or no vaccine.
Types of outcomes:
Example:
Primary outcome measures: Seroconversion, [that is], proportion of patients with
adequate anti-HBs response (10 IU/L or Sample Ratio Units).…Secondary
outcome measures: Adverse events of hepatitis B vaccinations…[and] mortality.
11/09/2021 pujasari@ui.ac.id
Methods: Data Extraction
Describe the method of data extraction.
Example:
We developed a data extraction sheet[,]…pilot-tested it on ten
randomly-selected included studies, and refined it accordingly. One
review author extracted the…data…and the second author checked
the extracted data. …Disagreements were resolved by discussion
between the two review authors.
Describe any processes used to obtain and confirm data from other
investigators.
11/09/2021 pujasari@ui.ac.id
Methods: Data Items
List and define all variables for which data were sought, using
the PICOTS framework as a guide.
List any assumptions and simplifications that were made in
defining the variables.
Example:
Information was extracted from each included trial on: (1)
characteristics of trial participants…and the trial’s inclusion and
exclusion criteria; (2) type of intervention…(versus placebo or
versus the type, dose, duration and frequency of another NSAID
[nonsteroidal antiinflammatory drug]; or versus another pain
management drug; or versus no treatment); (3) type of outcome
measure.
PICOT(S) = population, intervention, comparator, outcome, timing, and study design or setting
11/09/2021 pujasari@ui.ac.id
Methods: Quality Assessment
Describe the methods and criteria used to assess the quality (risk of bias)
of individual studies.
Specify whether or not the assessment was carried out at the study or
outcome level, or both.
Describe how this information is to be used in any data synthesis.
Examples from two separate studies:
Pairs of reviewers…determined the adequacy of randomization and
concealment of allocation, blinding of patients, health care providers,
data collectors, and outcome assessors; and extent of loss to follow-
up.
To explore variability in study results (heterogeneity) we
Tracz MJ, et al. J Clin Endocrinol Metab 2006;91:2011-6; Bucher HC, et al. BMJ 2000;321:73-7.
11/09/2021 pujasari@ui.ac.id
Methods: Synthesis of Results
11/09/2021 pujasari@ui.ac.id
Methods: Grading Strength of Evidence
Detail any assessment of risk of bias that may affect the cumulative
evidence.
Publication bias
Selective reporting within studies
Example:
For each trial we plotted the effect by the inverse of its standard
error. The symmetry of such “funnel plots” was assessed both
visually, and formally with Egger’s test, to see if the effect decreased
with increasing sample size.
11/09/2021 pujasari@ui.ac.id
Methods: Additional Analyses
Example:
• Sensitivity analyses were pre-specified. The
treatment effects were examined according to
Indicate which of these quality components (concealed treatment
allocation, blinding of patients and caregivers,
blinded outcome assessment), time to initiation
analyses were prespecified. of statins, and the type of statin. One post-hoc
sensitivity analysis was conducted including
unpublished data from a trial using
cerivastatin.
11/09/2021 pujasari@ui.ac.id
Results Overview
The results section of the review should contain the following subsections:
Study Selection
Study Characteristics
Quality Assessment
Individual Studies
Synthesis of Results
Grading Strength of Evidence
Additional Analyses
Sensitivity
Subgroup
11/09/2021 pujasari@ui.ac.id
Place at the beginning of the Results section, not in
Place the Methods section.
Results:
Study Give Give numbers of studies screened, assessed for
eligibility, and included in the review.
Selection
Give the reasons for exclusions at each stage of the
Give assessment, ideally illustrated with a flow diagram.
• Example:
11/09/2021 pujasari@ui.ac.id
Examples of Flow Diagrams
Sharma M, et al. Ann Intern Med 2009:151:622-30. Reprinted Fuccio L, et al. Ann Intern Med 2007;147:53-62. Reprinted
with permission from the American College of Physicians. with permission from the American College of Physicians.
11/09/2021 pujasari@ui.ac.id
Results: Study Characteristics
Study characteristics can be presented:
Within text
In summary tables and graphs
Example:
All four studies finally selected for the review were randomised
controlled trials published in English. The duration of the
intervention was 24 months for the RIO-North America and 12
months for the RIO-Diabetes RIO-Lipids and RIO-Europe study.
Although the last two described a period of 24 months during which
they were conducted, only the first 12-months results are provided.
All trials had a run-in, as a single blind period before the
randomisation.
11/09/2021 pujasari@ui.ac.id
Results: Quality Assessment
Present data on risk of bias for each study analyzed.
Present the results of outcome-level assessments, if available.
Example:
Devereaux PJ, et al. BMJ 2005;331:313-21, as adapted in Liberati A, et al. Ann Intern Med 2009;151:W65-
94.
11/09/2021 pujasari@ui.ac.id
Results: Individual Studies*
For all outcomes considered (benefits and harms), present the following
for each study:
simple summary data for each intervention group, and
effect estimates and confidence intervals (ideally with a forest plot).
Present the results of individual studies in evidence tables and summary
tables and not in the text.
Refer to the Presentation of Findings module to see examples of the tables
and graphs used to present summaries of individual studies.
* This may appear in the appendix for Evidence-based Practice Center reports.
11/09/2021 pujasari@ui.ac.id
Results: Synthesis of Results
Present synthesized results in text, summary tables, or evidence maps.
Text example:
Mortality data were available for all six trials, randomizing 311
patients and reporting data for 305 patients. There were no deaths
reported in the three respiratory syncytial virus/severe bronchiolitis
trials; thus our estimate is based on three trials randomizing 232
patients, 64 of whom died. In the pooled analysis, surfactant was
associated with significantly lower mortality (relative risk = 0.7, 95%
confidence interval = 0.4–0.97, P = 0.04). There was no evidence of
heterogeneity (I2 = 0%).
Summary tables: refer to the Presentation of Findings module
Evidence maps: refer to the Presentation of Findings
11/09/2021 pujasari@ui.ac.id
Results: Grading Strength of Evidence
Include risk of bias, directness, consistency, and precision in
reporting how evidence was graded.
Example:
There is a low level of evidence…that RFA [radiofrequency
catheter ablation] improves quality of life more than medical
treatment. Three RCTs [randomized controlled trials] and one
observational study reported more improvement in the general or
physical functioning…in patients who underwent RFA.…However,
these studies assessed the results at nonuniform time points and
therefore the findings may be difficult to interpret.
11/09/2021 pujasari@ui.ac.id
Results: Additional Sensitivity Analyses
Give the results of additional analyses, such as sensitivity or
subgroup analyses and meta-regressions.
Include the results of additional analyses to facilitate a better
understanding of heterogeneity.
Example 1:
[B]enefits of chondroitin were smaller in trials with adequate
concealment of allocation compared with trials with unclear
concealment (P for interaction = 0.050), in trials with an intention-
to-treat analysis compared with those that had excluded patients from
the analysis (P for interaction = 0.017), and in large compared with
small trials (P for interaction = 0.022).
11/09/2021 pujasari@ui.ac.id
Results: Additional Subgroup Analyses
Example 2:
Subgroup analyses according to antibody status, antiviral
medications, organ transplanted, treatment duration, use of
antilymphocyte therapy, time to outcome assessment, study
quality and other aspects of study design did not demonstrate
any differences in treatment effects. Multivariate meta-
regression showed no significant difference in CMV
[cytomegalovirus] disease after allowing for potential
confounding or effect-modification by prophylactic drug used,
organ transplanted or recipient serostatus in CMV positive
recipients and CMV negative recipients of CMV positive
donors.
11/09/2021 pujasari@ui.ac.id
Summarize the main findings, including the strength of
Summarize evidence for each main outcome.
Discussion:
Summary of Consider
Consider the applicability of findings to key groups (e.g.,
health care providers, users, and policymakers).
Evidence
Refer to the Assessing Applicability and Grading Strength of
Refer Evidence modules for additional guidance.
11/09/2021 pujasari@ui.ac.id
Discussion: Summary of Evidence
Example:
Compared with men who used watchful waiting , men with
clinically localized prostate cancer detected by methods other
than PSA [prostate-specific antigen] testing and treated with
radical prostatectomy experienced fewer deaths from prostate
cancer, marginally fewer deaths from any cause, and fewer
distant metastases. The greater benefit of RP on cancer-specific
and overall mortality appears to be limited to men under 65
years of age but is not dependent on baseline PSA level or
histologic grade.
11/09/2021 pujasari@ui.ac.id
Discussion: Limitations
Discuss the limitations of the review at different levels:
Study level (e.g., risk of bias)
Outcome level (e.g., benefits or harms)
Review level (e.g., incomplete retrieval of identified research;
reporting bias)
Example:
The meta-analysis reported here combines data across studies in
order to estimate treatment effects with more precision than is
possible in a single study. The main limitation of this meta-
analysis, as with any overview, is that the patient population, the
antibiotic regimen and the outcome definitions are not the same
across studies.
11/09/2021 pujasari@ui.ac.id
Discussion: Conclusions
Provide a general interpretation of the results in the context of other
evidence and the implications for future research.
Example:
[T]he available clinical trial evidence supporting the use of
combination therapies over higher dose statin therapy is insufficient
to guide clinical decisions. The long term clinical benefits and risks
of combination therapies have yet to be demonstrated. There are
some instances, such as failure to reach targets in spite of maximal
statin therapy, and populations with elevated triglycerides who need
to achieve secondary goals, in which clinicians may choose
combinations pending definitive evidence.
Sharma M, et al. AHRQ Comparative Effectiveness Review No. 16. Available at:
http://www.effectivehealthcare.ahrq.gov/ehc/products/11/171/reptbodyfin-typofixed4-12-2010.pdf.
11/09/2021 pujasari@ui.ac.id
Key Messages
Reporting a systematic review is the final step of the review process.
The report should convey in a transparent manner the methods and results
to readers, including consumers, clinicians, and policymakers.
Inadequate reporting makes it more difficult to judge the validity of the
methods and results.
11/09/2021 pujasari@ui.ac.id
THANK YOU
11/09/2021 pujasari@ui.ac.id