Metlit-08 Metode Penelitian SR-MA - RR

Competence and Credible, Excel in Organization, Excel in Operation, Beyond Expectations, Management by Objective
Metode Penelitian
Systematic Review/Meta-analysis
Indah S. Widyahening
Siti Rizny F. Saldi
Respati W. Ranakusuma
Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit
RSUPN Dr. Cipto Mangunkusumo – Fakultas Kedokteran Universitas Indonesia

Acknowledgements
 Paul Glasziou, Elaine Beller (IEBH, Bond University
Australia)
 Steve McDonald, Sally Green, Miranda Cumpston
(Cochrane Australasia)
 Cochrane Training
https://training.cochrane.org/online-learning
 Cochrane Handbook of Systematic Reviews of
Interventions
https://training.cochrane.org/handbooks

Challenges in clinical practice

Explanation of clinical profile: given the patient Diagnostic
profile, what the patient’s illness? knowledge
Explanation of illness: why did this illness occur in Etiologic

this patient at this time? knowledge
Prediction of course: given the patient’s illness, its’

aetiology, the clinical & non-clinical profile, etc., Prognostic
what will be the future course of the illness, knowledge
depending on (/absence of) treatment
Treatment decision Prognostic and

decision
Treatment execution analysis skill
Type of Article and Best study design

Type of Question Type of Study/Methodology
Therapy Information needed about Systematic review/Meta-
treatments (effectiveness, cost, analysis of RCTs
etc.) Double-Blind RCT
Diagnosis Information needed about a Systematic review/Meta-
diagnostic test (sensitivity, analysis of diagnostic studies
accuracy, etc.) Cross-sectional study with
random or consecutive sample
Prognosis Information needed about the Systematic review/Meta-
course of the disease over time, analysis of prognostic studies
expected complications, etc. Cohort/survival study
Aetiology/ Information needed about Systematic review/Meta-
Harm causes of disease or analysis of RCTs/obsv. studies
contributing factors of disease RCTs, cohort study, case-
control
RSUPN Dr. Cipto Mangunkusumo Study
– Fakultas Kedokteran Universitas Indonesia
Level of evidence
SR-MA
RCTs
Cohort studies
Case-controlled studies
Case series/reports
Background information/
expert opinion

Medical evidence increasing at epidemic rates:
we all need EBP skills to keep up-to-date
MEDLINE 2010
2,000 articles / day
Approximately
75 new trials
published every
day
Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?. PLoS Med. 2010;7(9)
So how should we go about summarizing
medical information?
How do we summarize medical information?
Traditional approach
 Expert opinion
 Narrative review articles
 Consensus statements
(group expert opinion)
New approach
(Systematic reviews)
 Explicit quantitative
synthesis of ALL the
evidence

Narrative/traditional reviews
(unsystematic)
Narrative/traditional reviews
(unsystematic)
What makes a review “Systematic”?

NARRATIVE
SYSTEMATIC
(OVERVIEW)
QUESTION Broad Focused
SOURCES/ SEARCH Usually unspecified; Comprehensive sources;
possibly biased explicit search stategy
SELECTION Unspecified; possibly Criterion-based;
biased uniformly applied
APPRAISAL Variable Rigorous
SYNTHESIS Often qualitative only Either
INFERENCE/ Seldom evidence-based Usually evidence-based
CONCLUSION
Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best
evidence for clinical decisions. AnnDr.Intern
RSUPN Med. 1997;126(5):376-80.
Cipto Mangunkusumo – Fakultas Kedokteran Universitas Indonesia
Systematic review
 “The application of strategies that limit bias in the
assembly, critical appraisal, and synthesis of all
relevant studies on a specific topic.”
Oxford Centre of Evidence Based Medicine (OCEBM) Levels Table
 Ensures that all available evidence is taken into

account and minimises ‘cherry-picking’
 Not performing SRs can be dangerous and/or

unethical!

Why Systematic Reviews?

 Information overload → Systematically search
 Not all studies in journals → Systematically assessed

are good in quality the quality of included
(valid/unbiased) studies
 Result from many studies → Systematically combined

not conclusive to arrive at conclusion
(controversies exist)

What is a Systematic Review?

 A Systematic Review is a review of a clearly formulated question
that uses systematic and explicit methods to identify, select and
critically appraise relevant research, and to collect and analyze
data from the studies that are included in the review.
 A systematic review is an overview of primary studies which

contains an explicit statement of objectives, materials and
methods and has been conducted according to explicit and
reproducible methodology.
 Two types:
 Qualitative (non-statistical)
 Quantitative (statistical) or meta-analysis

Why systematic reviews?
• efficient way to access the body of research
• saves time required for searching
• critical appraisal
• interpretation of results
• explore differences between studies
• reliable basis for decision making
• unbiased selection of relevant information
• useful for health care, policy, future research
Integrative Literature
Overview (review article)
Unsystematic
No sufficient appraisals
Prone to severe bias
Systematic review
Systematic
No formal statistical methods
Meta-analysis
Systematic review and formal
statistical methods

Integrative Literature
Overview (review article)
Unsystematic
No sufficient appraisals
Prone to severe bias
Systematic review
Systematic
No formal statistical methods
Meta-analysis
Systematic review and formal
statistical methods

What is meta-analysis?
 Statistical methods (meta-analysis) may or may not be used to
analyze and summarize the results of the included studies.
 Meta-analysis = quantitative approach for systematically
combining results of previous research to arrive at conclusions
about the body of research
 What does it mean?
 Quantitative → numbers
 Systematic → methodical
 Combining → putting together
 Previous research → what’s already done
 Conclusions → new knowledge
Considered as a true investigation

observational - retrospective study
Why meta-analysis?
 To quantitatively summarize estimate from
previous studies (i.e. resolve controversies)
 Helps guide for further research
 Increase statistical power
 Improves generalizability

• State the reasons why the review

Components of SR/MA is needed
• Based on problems in clinical
setting (high volume, high risk,
 Introduction: high cost)
• Any controversies in the
 Background
literatures? → In treatment,
 Research question(s) choice of diagnostic modalities,
 Objectives determination of prognosis
• Ended with objective of SR based
 Hypothesis on clinical question (PICO)
 Methods
• How the individual studies searched
 Results
and selected
 Discussion • How to appraise/assess the
individual studies
 Conclusions • How to combine (synthesis)

Steps in Systematic Review/Meta-Analysis

•Formulate question
A. Identification •Develop search strategy
•Searching relevant studies
•Select eligible studies

B. Selection
•Choose variables (comparison & outcomes)
•Assess quality
C. Abstraction •Extract data and calculate individual summary
measures
•Quantitative pooling
D. Analysis •Sensitivity analysis
•Investigate heterogeneity & biases
E. Write-up •Clear presentation

Steps of a Systematic Review following Cochrane Methods
1. define the question
2. plan eligibility criteria
3. plan methods
publish protocol
4. search for studies
5. apply eligibility criteria
6. collect data
7. assess studies for risk of bias
8. analyse and present results
9. interpret results and draw conclusions
publish review
10. improve and update review publish update
Steps of a Systematic Review following Cochrane Methods
1. define the question
2. plan eligibility criteria
Identification
3. plan methods
4. search for studies
5. apply eligibility criteria Selection
6. collect data
7. assess studies for risk of bias Analysis
8. analyse and present results
9. interpret results and draw conclusions
Analysis
10. improve and update review
https://www.crd.york.ac.uk/prospero/
https://www.equator-network.org/reporting-guidelines/prisma-protocols/
A. Identification
Be methodical: plan first → protocol
▪ Formulate your review questions

▪ Develop search strategy
▪ Searching relevant studies
▪ A comprehensive and reproducible literature
search is the foundation of a systematic review

How to search?
▪ Sources
▪ Find published relevant articles:
▪ Major bibliographic databases: PubMed/MEDLINE, EMBASE, Cochrane.
▪ Textbooks, printed journals, reference lists
▪ Unpublished materials:
▪ Theses, dissertations, trial registries, contact with experts/personal
communication (peer group), etc.
▪ Search strategies:
▪ Database: MeSH terms and text words
▪ Others: hand-searching → a manual page-by-page
examination of the entire contents, to identify all eligible
reports
▪ No limitation on years and languages
Literature search challenges

 Database Bias - “No single database is likely to contain
all published studies on a given subject.”
 English-language bias - occurs when reviewers exclude
papers published in languages other than English
 Citation bias - occurs when studies with significant or
positive results are referenced in other publications,
compared with studies with inconclusive or negative
findings
 Publication Bias - selective publication of articles that
show positive treatment of effects and statistical
significance.
Is finding all published studies enough?

 Negative studies less likely to be published than ‘Positive’
ones
 Editors tend to accept studies with positive than negative
results
 Positive results tend to be submitted to international
journals, whilst negative results are submitted to local
journals
 Many negative results are product of studies with small
sample size
 Inference by manufacturers

Identifying publication bias
Symmetrical funnel plot
0
1
Standard Error
3
0.1 0.33 0.6 1 3 10
Source: Matthias Egger & Jonathan Sterne Effect
Asymmetrical funnel plot
0
1
Standard Error
Small study
effect
2
3
0.1 0.33 0.6 1 3 10
Source: Matthias Egger & Jonathan Sterne Effect
Funnel Plots
Symmetrical plot in the
absence of bias (smaller
studies with no
significant
effect shown by open
circles)
Asymmetrical plot in the

presence of reporting
bias
Asymmetrical plot in the

presence of bias because
some smaller studies
(open circles) are of lower
methodological quality
of smaller studies
Other reasons for funnel plot asymmetry?
 True heterogeneity:
 If trials do not estimate the same underlying effect of the
same intervention
 Size of effect differs according to study size e.g.
if the true effect is larger in smaller trials
 Differences in underlying risk
 Intensity of intervention (comprehensive geriatric
assessment)
 Chance

To minimize publication bias:

(some) solutions
 All trials registered at inception in meta-registry of
trial registries
 www.clinicaltrials.gov
 www.controlled-trials.com
 Search in other sources:

 Conference proceedings
 Technical reports (Research, governmental agencies)
 Dissertations, theses
 Contact with primary researchers

B. Selection
Be methodical: plan first → protocol
Aim: to include all relevant studies, based on:
 Study design: randomization (for clinical trial), cohort, etc.
 Patients’ characteristics: diagnosis (including severity), gender, age,
group, race, etc.
 Similarity of exposure or treatment (e.g. drug class, dosage)
 Similarity of outcomes (case definitions)
 Setting (emergency department, outpatient, inpatient)
 Year of publication or conduct (especially if technology or typical dosing
changes)
 Minimum sample size
 Etc.

C. Abstraction
 Compile relevant data
 May need information from author(s)
 Summarize for analysis
 Analyze for risk of biases
 Weighted scoring: studies with large numbers &

more definite design weight more

How to abstract data

For each study, extract:
 name of the study
 name of the author, year published
 number of participants who received intervention
 number of participants who were in control arm
 number who developed outcomes in intervention
 number who developed outcomes in control arm
 All other relevant information referring to
inclusion/eligibility criteria

Abstraction - example
Bias assessment - scales and checklists

 many scales available
 not supported by empirical evidence
 different scales, different conclusions
 may include criteria not related to bias
 numerical weighting not justified
 difficult for readers to interpret the score

Cochrane ‘Risk of bias’ assessment

 7 evidence-based domains
 review authors’ judgement
✓ Low risk of bias
 High risk of bias
? Unclear
 support for judgement
 evidence/quotes from the paper or other sources
 review author’s explanation

Domains to address
 random sequence generation
 allocation concealment
 blinding of participants and personnel
 blinding of outcome assessment
 incomplete outcome data
 selective reporting
 other bias

Sources of bias
Random sequence generation Target Population
Selection
Allocation concealment Allocation
Blinding of Intervention group Control group

Performance
participants, personnel
Blinding of outcome
Detection
assessment Outcome Outcome
assessment assessment
Attrition Incomplete outcome data
Reporting Selective reporting Publication of study outcomes

Risk of bias assessment
Risk of bias assessment
D. Analysis
 Combine data to arrive at a summary, 3 measures
 Effect Size (OR/RR/MD), with CI of each study
 Variance with 95% Confidence Interval
 Test of heterogeneity
 Two Graphs
 Forest Plot
 Funnel Plot
 Examine why the studies are heterogeneous (if
they are) → test of heterogeneity
 Use Statistical Packages, several choices

Study level Review level
↓ ↓
Study A Outcome data Effect measure
Study B Outcome data Effect measure
Effect measure
Study C Outcome data Effect measure
Study D Outcome data Effect measure
Source: Jo McKenzie & Miranda Cumpston

What is Forest Plot?

Understanding evidence: The big picture
Systematic reviews are the best way to

understand the effects of treatments because they
consider all the relevant, reliable evidence.
These slides will teach you how to interpret the

main graph used to present the results of a
systematic review of randomized trials.
Forest Plot Favours Treatment A Favours Comparator B
Main Graph
Trial 1
Trial 2
Trial 3
Trial 4
Many similar trials
Trial 5
Trial 6
Trial 7
Trial 1+2+3
+4+5+6+7
Relative Risk (RR)

Steroids given before preterm delivery
reduce the number of babies who die.
Favours steroid Favours placebo
percent who died with steroid

RR = 11.2%
percent who died with placebo "No difference"
RR=1
10%
5% die
10% die with
with steroid
steroid
10%
5% die
10% die with
with placebo
placebo
RR
RR =
= ??
??
0.01 0.1 0.5 1 2 10
Relative Risk
Relative Risk of Death
(RR)
(RR) of death
Trial Deaths
Deaths
Babies Steroid Placebo
#1 1070 6.8
36 % 11.2%
deaths 36 deaths
532 babies 532 babies
6.8%
percent who died with steroid
RR = = 0.61
11.2%
11.2%
percent who died with placebo
0.01 0.1 0.5 1 2 10
Relative
Relative
Risk
Risk (RR)
(RR) of Death
of death
Favours Point
steroid Favours placebo
estimate
Trial Deaths
Babies Steroid Placebo "best guess at truth"
#1 1070 6.8% 11.2%
95% Confidence interval

"truth probably in here"
0.01 0.1 0.5 1 2 10
Relative
Relative
Risk
Risk (RR)
(RR) of Death
of death
Trial Deaths
Babies Steroid Placebo "Cumulative Meta-analysis"
#1 1070 6.8% 11.2%
#2 130 1.4% 8.2%
#3 126 3.0% 11.9%
#4 127 9.0% 10.0%
#5 146 1.4% 6.7%
#6 144 2.5% 15.9%
#7 743 9.7% 10.0%
Total 2486
0.01 0.1 0.5 1 2 10
Relative
Relative
Risk
Risk (RR)
(RR) of Death
of death
0.1 0.5 1 2 10
Forest plot
The label tells you what the comparison and outcome of

interest are
Forest plot
Each study has an ID

Forest plot
The data for each trial are here, divided into experimental and
control groups
Forest plot
• This is the % weight given to each study in the pooled analysis
• For the mean difference: SDs are used together with the sample
sizes to compute the weight given to each study.
Forest plot
Results from each trial are also given numerically

Forest plot
The label above the graph tells you which statistic has been
used
Forest plot
• Mid point of the box shows point effect estimate of each study
• The size of the block for each study is proportional to the % weight.
• The horizontal line represents the confidence interval.
Forest plot
• Take care to read the labels
• Things to the left do not always mean the treatment is
better than the control
Forest plot
• The pooled analysis is given a diamond shape
• The widest bit is located on the point estimate, and the horizontal width in the
confidence interval.
• Does the confidence interval cross the line of no effect?
Overall effect
estimate
Forest plot
Results of the pooled analysis are also given numerically

Forest plot
• Test for heterogeneity
• The result of a Z test with a p value are also given for the overall
effect
1. Which is the largest trial?
2. Which is the smallest trial?
3. How many trials favour treatment?
4. How many trials are statistically significant?
1
2
3
4
5
Study results (1)

 For binary/nominal outcomes:
 Example: death, relapse, cure, may be continuous
originally (e.g. pain relief)
 Presented as OR for individual study with corresponding
95% CI
 Represented by square, the area = relative weighted
value
 Summarized as combined OR with CI as diamond
 Vertical line → OR = 1 (no effect)
 Vertical dotted line → if CI crosses this line means
homogeneous, otherwise heterogeneous
Meta-analysis: Example of graph presentation of

RCTs with nominal outcome
Study I 1992
Study II 1994
Study III 1995
Study IV 1995
Study V 1996
Study VI 1997
Study VII 1999
Study VIII 2000
Combined
0.1 10
OR = 1
Favor drug Favor placebo
Study results (2)

 For continuous/numeric outcomes:
 Example: volume (e.g. blood loss), quality of life, physical
measurement (e.g. lung function), duration (e.g. stay in
hospital)
 Presented as mean difference for individual study with
corresponding 95% CI
 Represented by square, the area = relative weighted value
 Summarized as combined mean difference with CI as diamond
 Vertical line → mean diff = 0 (no effect)
 Vertical dotted line → if CI crosses this line means
homogeneous, otherwise heterogeneous

Meta-analysis: Example of graph presentation of

RCTs with numerical outcome (xe-xc)
Study I 1992
Study II 1994
Study III 1995
Study IV 1995
Study V 1996
Study VI 1997
Study VII 1999
Study VIII 2000
Combined
-0.3 +0.3
Favor drug Xe- Xc= 0 Favor placebo
Issues in meta-analysis
 Choosing a model -- heterogeneity
 Fixed effects model or random effects?
 Bias in meta analysis
 poor quality of trials
 publication bias
 Quality control in meta analysis
 PRISMA guidelines
 Statistical Software for meta analysis

Looking for causes of heterogeneity

What is heterogeneity
Variation or Differences
between the true intervention effects underlying
the different studies
Differences across studies in meta-analysis.

Studies should have similar participants,
interventions, comparisons, and outcomes
Clinical Methodological Statistical

Clinical diversity/heterogeneity
 Participants
 e.g. condition, age, gender, location, study eligibility
criteria
 Interventions
 intensity/dose, duration, delivery, additional
components, experience of practitioners, control
(placebo, none, standard care)
 Outcomes
 follow-up duration, ways of measuring, definition of an
event, cut-off points

Methodological diversity/heterogeneity
 Design
 e.g. randomised vs non-randomised
 Conduct
 e.g. risk of bias (allocation concealment, blinding, etc.),
approach to analysis

How to deal with heterogeneity

 Ignore it and use fixed effects model (not
recommended)
 Test for it and do not pool results if studies are
significantly heterogeneous
 Incorporate it:
 assume that heterogeneity is due to random differences
among studies whose sources can not be identified
 use random effects model

Identifying heterogeneity
 Visual inspection (eyeball test) of forest plots
 Chi-squared (χ2) test (Q test)
 I2 statistic to quantify heterogeneity

The chi-squared (χ2) test (1)

 The Chi-squared (χ2) is a statistical test for
heterogeneity which assesses whether observed
differences in results are compatible with chance
alone
 The consistency of studies’ results → if CI for the
results of individual studies have poor overlap,
could indicates the presence of statistical
heterogeneity


 Low power in small size studies or few in number
 while a statistically significant result may indicate a
problem with heterogeneity, a non-significant result
must not be taken as evidence of no heterogeneity (p-
value 0.10 → to determine statistical significance)
 High power in many studies in a meta-analysis
 may detect clinically unimportant differences with many
studies


The I2 statistic (1)

 I2 statistic describes the percentage of variability due to
heterogeneity rather than chance (0%-100%)
 Low values indicate no, or little heterogeneity
 0%-40%: might not be important
 High values indicate a lot of heterogeneity
 30%-60%: may represent moderate heterogeneity
 50%-90%: may represent substantial heterogeneity
 75%-100%: considerable heterogeneity
 Be cautious in interpreting: can be misleading since the

importance of inconsistency depends on several
factors

The I2 statistic (2)

Analysis
 Many techniques
 Focus on variability
 Fixed effects model
 Only intra-study, ignore inter-study variability
 Narrower CI
 Random effects model
 Considers also inter-study variability
 Broader CI
 Bayesian statisticians (less popular)
 Prior probability
 Even broader CI
 Unless the studies are very heterogeneous → fixed and
random model usually similar

Analysis: fixed effect vs random effect

 Two models for meta-analysis available in RevMan
 Make different assumptions about heterogeneity
 Pre-specify your planned approach in your protocol
 fixed-effect
 may be unrealistic – ignores heterogeneity
 random-effects
 allows for heterogeneity
 estimate of distribution of studies may not be accurate if biases
are present, few studies or few events

Fixed-effect model
Random (sampling)
error • assumes all studies are
measuring the same treatment
effect
• estimates that one effect
• if not for random (sampling)
error, all results would be
identical
Study
result
Common
Source: Julian Higgins true effect
Random-effects model
Random
error • assumes the treatment effect
varies between studies
• estimates the mean of the
distribution of effects
• weighted for both
within-study and between-study
variation (tau2, 2)
Study-
specific
effect
Mean of true
Source: Julian Higgins effects
What’s the difference?

Random-effects meta-analyses are:
 almost identical to fixed-effect when there is no
heterogeneity
 similar to fixed-effect but with wider confidence
intervals when there is heterogeneity of the sort
assumed by RE model
 different from fixed-effect meta-analyses when
results are related to study size
Random-effect model gives relatively more weight to smaller studies.

It’s recommended to run your meta-analysis both ways and see how the results differ.
No heterogeneity
Fixed Random
Adapted from Ohlsson A, Aher SM. Early erythropoietin for preventing red blood cell transfusion in preterm
and/or low birth weight infants. Cochrane Database of Systematic Reviews 2006, Issue 3.
Some heterogeneity
Fixed Random
Adapted from Adams CE, Awad G, Rathbone J, Thornley B. Chlorpromazine versus placebo for schizophrenia.
Cochrane Database of Systematic Reviews 2007, Issue 2.
Small study effects
Fixed Random
Adapted from Li J, Zhang Q, Zhang M, Egger M. Intravenous magnesium for acute myocardial infarction.
Cochrane Database of Systematic Reviews 2007, Issue 2.
Investigating heterogeneity
 Does the intervention effect vary with

different populations or intervention
characteristics (such as dose or duration) →
Interaction and effect modification
 Methods to identify this:
 Subgroup analysis
 Meta-regression

Subgroup analysis
 Splitting all the participant data into subgroups to
make comparisons between them
 Aims:
 To investigate heterogeneous results
 To answer specific questions about particular patient
groups, types of intervention or types of study.
 Example: Subsets of participants (males and
females), study settings (different geographical
locations).

Meta-regression
 If studies are divided into subgroups, this may be viewed as an
investigation of how a categorical study characteristic is
associated with the intervention effects in the meta-analysis
 For example, studies with adequate allocation sequence
concealment may yield different results from inadequate
concealment → Allocation sequence concealment (adequate vs.
inadequate) is a categorical characteristic at the study level.
 Definition: An extension to subgroup analyses that allows the
effect of continuous, as well as categorical, characteristics to be
investigated, and in principle allows the effects of multiple
factors to be investigated simultaneously (although this is rarely
possible due to inadequate numbers of studies) (Thompson and
Higgins 2002).
Meta-regression should not be considered when

there are < 10 studies in a meta-analysis. Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit
Selection of study characteristics for

subgroup analysis and meta-regression
 Adequate number of studies
 Pre-specify characteristics in the protocol
 Select a small number of characteristics
 Scientific rationale foe investigating each
characteristics

Sensitivity analysis
 Most decisions in systematic review are clearly objective and
non-contentious, BUT some are unclear.
 We use sensitivity analysis to prove that the findings from a
systematic review are not dependent on such arbitrary/unclear
decisions
 A sensitivity analysis is a repeat of the primary analysis/meta-
analysis in which alternative decisions or ranges of values are
substituted for decisions that were arbitrary or unclear.
 Example:
If the eligibility of some studies in the meta-analysis is dubious because
they do not contain full details, sensitivity analysis may undertake the
meta-analysis twice
 First, including all studies
 Second, including only those that are definitely known to be eligible.
Sensitivity analysis is not sub-group analysis Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit
Potential domains for sensitivity analysis

 Searching for studies
 Eligibility criteria
 Characteristics of P – I – C – O – S
 Age range (participant), range of dose of X drug (intervention), definition of usual
care (comparator), time point are eligible for inclusion (outcome), blinded vs
unblinded outcome assessment (study design)
 Data for analysis
 Continuous data (missing SDs, change score vs post-intervention values)
 Ordinal scales (cut-point value to dichotomize short ordinal scales into 2
groups)
 Others (time-to-event data, cluster-randomised and crossover trials)
 Analysis methods:
 Fixed-effect vs. random-effect methods
 Dichotomous outcomes: OR vs. RR vs. risk difference?
 Continuous outcomes using several scales: standardised mean
difference vs. individual mean difference
Confounders in non randomised studies

(NRS)
 Different decisions among researchers concerning
confounding factors
 It requires a method to control for confounding, as an
important source of heterogeneity between studies, such
as:
 Many (but not all) NRS describe the confounding factors
that were considered and whether confounding was taken
into account by the study design or analysis; most also
report the baseline characteristics of the groups being
compared
 Assessing what researchers actually did to control for
confounding may be difficult; far fewer studies describe
precisely how confounding factors were measured or fitted
as covariates in regression models

Confounders in non randomised studies

 Suggestions for assessing confounders related to
selection bias:
 list potential confounding factors in the protocol (based
on evidence, expert opinion from members of the review
team and advisors)
 Identify the confounding factors that the researchers
have considered and those that have been omitted.
 Assess the balance between comparator groups at
baseline with respect to the main prognostic or
confounding factors.
 Identify what researchers did to control for selection bias
(i.e. any design features, analysis methods)

E. Write Up
https://www.equator-network.org/reporting-guidelines/prisma/

Advantages of systematic reviews

 Explicit methods limit bias in identifying and rejecting studies
 Conclusions are more reliable and accurate because of methods
used
 Large amounts of information can be assimilated quickly by
healthcare providers, researchers, and policymakers
 Delay between research discoveries and implementation of
effective diagnostic and therapeutic strategies may be reduced
 Results of different studies can be formally compared to
establish generalisability of findings and consistency (lack of
heterogeneity) of results
 Reasons for heterogeneity (inconsistency in results across
studies) can be identified and new hypotheses generated about
particular subgroups
 Quantitative systematic reviews (meta-analyses) increase the
precision of the overall result
Meta-analysis: Advantages & disadvantages
Advantages:
▪ Combined effect size: more definitive
▪ Enables subgroup analysis
▪ Background for further studies
Disadvantages:
▪ Statistical controversies
▪ Publication bias
▪ Disadvantages of secondary data
▪ Technical limitations: Details of studies rarely identical

Key points to remember

 Systematic reviews assemble, critically appraise, and
synthesize the results of primary investigations
addressing a specific topic or problem
 Systematic reviews are prepared using strategies that
limit bias and random error
 Systematic reviews are integrative articles;
 Systematic reviews can help practitioners keep up to
date with the overwhelming volume of medical
literature
 Systematic reviews can help ground clinical decisions
in research evidence, although they neither make
decisions nor obviate the need for sound,
compassionate clinical reasoning
Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of best
evidence for clinical decisions. AnnDr.Intern
RSUPN Med. 1997;126(5):376-80.
Cipto Mangunkusumo – Fakultas Kedokteran Universitas Indonesia
Thank you


Metlit-08 Metode Penelitian SR-MA - RR

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Metlit-08 Metode Penelitian SR-MA - RR

Uploaded by

Copyright:

Available Formats

Competence and Credible, Excel in Organization, Excel in Operation, Beyond Expectations, Management by Objective

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Challenges in clinical practice

Explanation of illness: why did this illness occur in Etiologic

Prediction of course: given the patient’s illness, its’

Treatment decision Prognostic and

Type of Article and Best study design

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

How do we summarize medical information?

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

What makes a review “Systematic”?

 Ensures that all available evidence is taken into

 Not performing SRs can be dangerous and/or

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Why Systematic Reviews?

 Not all studies in journals → Systematically assessed

 Result from many studies → Systematically combined

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

What is a Systematic Review?

 A systematic review is an overview of primary studies which

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Considered as a true investigation

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

• State the reasons why the review

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Steps in Systematic Review/Meta-Analysis

•Select eligible studies

E. Write-up •Clear presentation

▪ Formulate your review questions

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Literature search challenges

Is finding all published studies enough?

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Asymmetrical plot in the

Asymmetrical plot in the

Other reasons for funnel plot asymmetry?

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

To minimize publication bias:

 Search in other sources:

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

 Weighted scoring: studies with large numbers &

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

How to abstract data

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Bias assessment - scales and checklists

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Cochrane ‘Risk of bias’ assessment

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Blinding of Intervention group Control group

Reporting Selective reporting Publication of study outcomes

Clinical Epidemiology and Evidence-Based Medicine (CEEBM) Unit

Study A Outcome data Effect measure

Study B Outcome data Effect measure

Study C Outcome data Effect measure

Study D Outcome data Effect measure

Source: Jo McKenzie & Miranda Cumpston

What is Forest Plot?