Journal club

Presenter : Dr. Abhishek Kumar

Moderator : Dr. Bobbity Deepthi
• Journal: Pediatric nephrology (Journal of the International pediatric
nephrology association).

• Date of publication: 03 January 2022.

Evidence based pyramid: Quality of evidence
 Globally published
literature

Presented as
final result back
to globe.

Comprehensive Appraised and

search for synthesized
relevant studies according to
predetermined
methods
 Steps in systematic review:

Analyse
and
Research Searching Quality present
question literature assessment results

PUBLISH

Eligibility Select Extract Interpret

criteria: studies. data result and
Inclusion draw
and conclusion
exclusion
criteria
Types of literature review:

• Authors opinion based review on a given area of interest.

Narrative review • Present data on a piece of argument

Systematic • Comprehensive literature search.

• Descriptive presentation of collected data.
review • Used for heterogenous data.

• One outcome of systematic review.

Meta-analysis • Summarize DATA using statistical(quantitative) method.
• Needs homogenous data: similar study group, outcome, etc.
 Types of systematic review:

Intervention review • Data from clinical trial.

Systematic reviews of • Data from cross-sectional studies, cohort

observational studies and case control

• Diagnostic accuracy test.

Diagnostic review • Can be any type of study.
• Journal: Pediatric nephrology (Journal of the International pediatric
nephrology association).

• Date of publication: 03 January 2022.

Rationale of the study:

• Early recognition of patients at risk for severe kidney injury (AKI) by

renal angina index (RAI) may help in the early institution of
preventive measures.
METHODOLOGY
Study Design:
• Systematic review with meta-analysis
Eligibility criteria:
• Inclusion criteria of studies:
1. Cohort and cross-sectional studies.
2. Children less than 18 years old.
3. Index test: RAI alone (calculated on day 0) or in combination with
biomarkers.
4. Reference test for defining AKI: Serum creatinine
5. Target disease: AKI as defined by KDIGO criteria or equivalent criteria.
6. Study design: diagnostic test accuracy studies.

• Exclusion criteria:
1. Case control studies.
2. Studies enrolling patients with kidney transplantation or chronic kidney
disease.
Objectives
• Primary objective:
• To assess the diagnostic accuracy of RAI on Day 0 of admission in predicting severe
AKI (stage 2 and 3 as per KDIGO guidelines or its equivalent stage, that is, injury and
failure in pRIFLE criteria) on day 3 of hospitalization.

• Secondary objective:
• Diagnostic accuracy of RAI in predicting any stage AKI on day 3 and receipt of KRT.
• Diagnostic accuracy of a combination of RAI and novel biomarkers in predicting
severe AKI was also assessed.
• Effect of RAI positivity (RAI >= 8) on mortality, ICU stay, and mechanical ventilation
duration.
Information sources and search strategies:
• PubMed Keywords used:
Acute kidney injury,
• EMBASE Pediatrics,
Studies published till may 2021 Adolescent,
• Web of Science Renal angina index, and
Biomarkers.
• CENTRAL

• Proceedings of relevant conferences and references of included studies

were scrutinized too.
Quality Assessment:
• Two investigators (JM, JK) independently assessed the quality of each
study.

• Validated tool used: QUADAS- 2 (Quality Assessment of Diagnostic

Accuracy Studies- 2).
• Analyse strength of evidence and imbedded biases.
• If found poor in evidence/ presence of bias: skewed data.
• Acknowledge it or exclude it.
Statistics used:
• Bi-variate model for meta-analysis as the index case had a uniform cut-off (>=
8) in all studies.

• Bivariate model: logit transformed sensitivity and specificity were modelled

jointly along with its correlation.

• Summary ROC curve were generated with 95% confidence region and
prediction region.
Statistics used:
• Heterogeneity was assessed by
• examining forest plot and using Chi2 and I2 statistics.
• Meta-regression analysis for predefined covariates: study design, setting, number of
sites and risk of bias. To examine for their effect on diagnostic accuracy.

• Sensitivity analysis performed for risk of bias of studies.

• Reporting bias: evaluated using Deek’s test.
• Publication bias:
• Fagon plot: to assess post-test probability.
• GRADE approach: to assess certainty of evidence for various outcome.
RESULTS:
 Study selection and characteristics:

IDENTIFICATION

SCREENING

22 studies: 24 reports:
INCLUDED 14001 children
 Studies: 22 ( prospective studies: 17; retrospective studies:
4; mixed design: 1)
Reports: 24.

Number of children evaluated: 14,001.

FINAL RAI on day 0 with cut-off >=8: RAI positivity.

DATA 19 studies used KDIGO criteria; 3 used pRIFLE criteria.

Any stage AKI: 35%

Severe AKI: 15%

DETAILED CHARACTERISTICS:
 How much variability is
How to there in the studies ?
interpret
the results What are the pooled
of a results ?
systematic What is a single pooled
review ? outcome ? (Meta-analyses)
Primary outcome: Forest plot.
Heterogeneity:
• Variability among the included studies in analysis is called variability.
• Parameters: study size, intervention, outcomes, study type.
• Variation within a parameter of different studies: statistical
heterogeneity a.k.a statistically different. Represented by p value <
0.05.
• Methods: Forest plot (eye-balling), chi-square test, I2 statistic, ROC
curve.
• Higgins I2: >50% is substantial heterogeneity.
In the current review -attempt to overcome
heterogeneity:
• Sensitivity analysis: removed outliers by excluding neonates or
patients aged >18 years.

• No improvement noted.
• Sensitivity: 0.88 (0.81 – 00.93, I2 = 74%)

• Specificity: 0.77 (0.68 – 0.85, I2= 97%)

Summary ROC curve: metaregression
ROC:
• AUC reflects overall performance of the test.
• Perfect test is AUC of 1.
• Can be used to compare accuracy of different diagnostic test.
Confidence interval vs prediction interval:

• Confidence Intervals are estimates that are calculated from sample

data to determine ranges likely to contain the population
parameter(mean, standard deviation) of interest.

• The range that likely contains the value of the dependent variable
for a single new observation given specific values of the
independent variables, is the prediction interval.
Primary outcome: RAI in predicting severe
AKI on D3.
• 18 studies (n= 5847) reported severe AKI.
• Sensitivity: 0.86 (0.77 to 0.92)
• Specificity: 0.77 (0.68 to 0.83)
• AUC: 0.88 (0.85 to 0.91)
• Diagnostic odds ratio: 21 (12-37)
• Higgin I2: substantial heterogenity. (Sn: 94% and Sp: 97.5%)
• sROC: substantial heterogenity.
Statistical methods:
Diagnostic Performance Of Renal Angina
Index
Odds ratio:
• Strength of association between an exposure and an outcome.
• The odds that an outcome will occur given a particular exposure,
compared to the odds of the outcome occurring in the absence of that
exposure.
EXPOSED

OUTCOME

NOT
EXPOSED
Diagnostic Odds Ratio:
• A measure of the effectiveness of a diagnostic test.

RAI >= 8

AKI

RAI < 8

• ranges from zero to infinity, although for useful tests it is greater than one,
and higher diagnostic odds ratios are indicative of better test performance.
Pre and post-test probability:
• Pre: 15% (based on pooled prevalence from included studies)
• Post-test probability of severe AKI:
• Positive: 40%
• Negative: 3%
Likelihood ratio:
• Tells if a diagnostic test will be informative in a given scenario.
• Also tell “how likely that patient has a disease”.
Interpretation of likelihood ratio: Fagan plot
Fagan plot:
Publication bias:
• When research studies are not published due to insignificant findings,
small effect size, unfavourable outcomes.

• Can cause of overestimate or underestimate the pooled estimate.

• Meta-analysis uses funnel plot
to help detect publication bias.

• Based on precision of the data

result.

• If no bias: the funnel plot will

almost symmetrical and
inverted funnel shape.
Publication bias: Deek’s test: p =0.01: present.
Secondary outcomes
Subgroup analysis:
• Sepsis: (4 studies, 833 participants)
• Sensitivity: 0.93 (0.79 – 0.98, I2 = 76 %), I2 = 76%
• Specificity: 0.58 (0.42 – 0.72, I2 = 92%), I2 = 92%

• Heterogenous group(non-sepsis group)

• Sensitivity: 0.84 (0.72 – 0.91, I2 = 94 %)
• Specificity: 0.81 (0.73 – 0.87, I2 = 97%)
Secondary outcome: combination of novel
biomarkers and renal angina index.
Other outcomes: mortality association (11
studies, 4958 participants)
Other outcome: ICU stay

• Mean difference: 1.6 days with 95%CI (0.2 – 0.3)

• I2 = 84%.
Other outcome: required mechanical
ventilation.

• Mean difference: 2.1 days with 95% CI(0.23 – 4.0)

• I2 = 92%
Certainty of evidence assessed: GRADE
approach
Overall summary:
DISCUSSION
For AKI prediction:
• RAI alone and RAI with urinary NGAL positivity on day 0 has good
diagnostic performance in predicting severe AKI on day 3. (moderate
certainty).
• RAI with urinary NGAL has better specificity than RAI alone in
predicting severe AKI on day 3.
• RAI with other biomarkers: variable result.

• RAI has good diagnostic performance in predicting any stage AKI on

day 3 as well as receipt of KRT. (moderate certainty).
Complications:
• RAI positivity is a/w longer ICU stay, longer duration of mechanical
ventilation and higher mortatlity.
Previous 2 meta-analysis:
Index study:
STRENGTH
• Risk of bias assessment done.
• Duplicate datas were excluded.
• Certainty of evidence for critical
outcome (GRADE approach)
LIMITATIONS
• Substantial heterogeneity.
• Attempted sub-group analysis
and meta-regression. [failed]
• Unexplained.
• p/o reporting bias.
• Applicability concern in 7 studies
in patient selection group.
Clinical utility: hypothetical cohort of 1000
critically sick child. Prevalence of AKI: 15%.
• RAI: sensitivity: 0.86 and specificity: 0.77.
• Calculate false positives and false negatives.

• FN: 21(2.1%): low  well accepted in clinical practice.

• FP: 196(19.6%): high  no harm in taking active measures among low
risk group.
CONCLUSION:
• With moderate certainty evidence, RAI >= 8 has good predicting
ability to recognize children at risk of AKI on day 3 and receipt of KRT.

• With low certainty evidence: combination of RAI and urinary NGAL

further improves the predicting ability.

• More studies on biomarkers in early prediction of AKI in RAI-positive

patients.
Question
Did the review address a clearly focused
question?
Did the authors look for the right type of
papers?
Do you think all the important, relevant
studies were included?
Did the review’s authors do enough to
assess quality of the included studies?
If the results of the review have been
combined, was it reasonable to do so?
Comment Yes No Can’t
tell
P I C O were defined
Major databases were used.
Quality of studies included were
assessed with appropriate tools
Results across studies were similar
Fixed effects model was used
Outcome is AKI (severe/ any stage)
Combining results was appropriate
Question Comment Yes No Can’t
tell

What are the overall results of the review?
How precise are the results?
Results were expressed as pooled
sensitivity and specificity with AUC
expressing strength of association.
Not precise due to outliers and
presence of bias.

Can the results be applied to the local Yes

population?

Were all important outcomes considered? Yes

Are the benefits worth the harms and Yes

costs?
THANK YOU!!
TERMINOLOGIES:
• Types of variable: independent, dependent. Covariate.
• Meta-regression
• Likelihood ratio and Pre and post-test probability.
• SENSITIVITY AND SPECIFICITY