Evidence-based medicine (EBM)

Marwan Adwan 07/08/2008

Evidence-based medicine (EBM)

The process of:

systematically

reviewing, appraising and using clinical research findings to aid the delivery of optimum clinical care to patients

Reviewing Research findings:

Where do we go? Electronic / printed Which database? Which journal? Which articles?

What do we do with the research once we retrieve it?

CRITICAL APPRAISAL

Why Critical appraisal?

Scenario
You are in a busy clinic, A patient shows you a newspaper clipping, reviewing a recent article from a national medical journal that warns against the use of HRT because of links with breast cancer. You assess her risk of breast cancer as low, but she declines HRT. When you discuss with her the results of an article showing that postmenopausal use of oestrogen reduces the risk of IHD, she counters with another article that concludes that cardiovascular mortality is increased in oestrogen users!

Why appraisal?

Not all RCTs & SRs are necessarily of good quality & therefore should be appraised and not simply trusted unquestionably.

HOW DO WE CRITICALLY APPRAISE A PAPER?

CONSIDER 3 ISSUES:
Validity } Reliability } 11 QUESTIONS Applicability}

A1. validity
1.

Did the study ask a clearly focused question in terms of:

Population Intervention Outcome

2. 3.

Randomised? Were all the subjects who entered the trial properly accounted for at its conclusion?

 YES?

continue Appraising

NO?

not worth reading

A2. Validity
Are there any differences between the two groups which could explain the differences between them? (age, sex, social class)

Consider:
4. Blinding 5. Were the groups similar at the start of the trial? 6. Apart from the intervention, were the groups treated equally?

B. What are the results? (RELIABILITY)

7. How large was the difference between the two groups?

8. How precise was the estimate of the treatment effects?

C. Will the results help locally? (APPLICABILITY)

9. Can the results be applied to your local population? 10. Were all the important outcomes considered? 11. Are the benefits worth the harms and costs?

A1. validity
1.

Did the study ask a clearly focused question in terms of:

Population Intervention Outcome

2. 3.

Randomised? Were all the subjects who entered the trial properly accounted for at its conclusion?

Population:
Residence, Age, Gender, Occupation How were they recruited? Who was included/excluded

SELECTION BIAS!

Exclusion of people with mild / severe disease misleading conclusions about treatment effect

Intervention:
Is dose adequate? Is duration adequate? What is the comparison? Placebo?

outcome
What are the outcomes? measured over what period? Is the outcome measure valid?

Systematic BIAS

Anything that erroneously influences the conclusions about groups and distorts comparisons

Fig 1 Sources of bias to check for in a randomised controlled trial

Greenhalgh, T. BMJ 1997;315:305-308

Copyright 1997 BMJ Publishing Group Ltd.

Other BIAS

Attrition bias

Publication bias Language bias MD bias

Randomisation

Good:

random number generator random number tables Alternate days / months Date of birth

Bad:

Were all the subjects who entered the trial properly accounted for at its conclusion?

Original and Proposed Revised Versions of CONSORT Flow Chart

Egger, M. et al. JAMA 2001;285:1996-1999.

Copyright restrictions may apply.

Egger, M. et al. JAMA 2001;285:1996-1999.

"intention to treat" analysis

were patients analysed in the groups to which they were randomised?

A2. Are there any differences between the two groups which could explain the differences between them? (age, sex, social class)

Consider:
4. Blinding 5. Were the groups similar at the start of the trial? 6. Apart from the intervention, were the groups treated equally?

Blinding
Participants Health care workers Study personnel

Were the groups similar at the start of the trial?

Baseline demographics

Apart from the intervention, were the groups treated equally?

Frequency of being seen Depth of examination Blood tests etc.

PERFORMANCE BIAS

B. What are the results?

7. How large was the difference between the two groups?

8. How precise was the estimate of the treatment effects?

Results:
Sample size calculation ARR RRR Survival analysis

Sample size
Power What is the right size?

Depends

how confident you want to be about results & how much error you are willing to accept

Too large = waste of money &

Too small = results cannot be generalised

Type 1 & 2 error

Type 1 error: assuming a relationship exists when it doesnt (Smoking causes Parkinsons)

Type 2 error: assuming no relationship exists when in fact it does (Smoking does not cause lung ca)

ARR

ARR = event rate in placebo event rate in treatment group

Eg

in a study of a drug to prevent MI 18.5% on placebo had MI & 9% on treatment had MI ARR = 18.5 - 9 = 9.5%

RRR

RRR =

event rate in placebo event rate in treatment group event rate in placebo = 18.5 9 18.5 = 9.5 18.5 = 51%

How precise was the estimate of the treatment effects?

P Value
The probability that the results observed in a study could have occurred by chance <0.05 statistically significant The smaller the better

Confidence intervals

95% CI: range covered by 2SD above & below mean

CI

= mean +/- 1.96SD

99% CI = mean +/3SD

C. Will the results help locally?

9. Can the results be applied to local population?

10. Were all the important outcomes considered? 11. Are the benefits worth the harms and costs?

9. Can the results be applied to local population?

Are there are any differences between the participants in the trial and the local population that would make it impossible to apply the results locally?

Any variable? Eg:

Hollywood stars all participants from inland Europe Migrants etc.

10. Were all the important outcomes considered?

Has the paper answered the original research question and whether any other important outcomes have been highlighted or missed out

example
Symptoms ADL Hospitalisation Adverse effects

11. Are the benefits worth the harms and costs?

NNT

the number of patients you need to treat to prevent one additional bad outcome NNT = 1 . ARR

= 1/ 9.5% = 10

Hypothetical drug A
NNT of 60 Bd dose 400 per dose Tratment for 12 months Total cost = 400 x 2 x 360 x 60 = 17.3M to save one life WOULD YOU PRESCRIBE IT?

Hypothetical drug B
NNT of 3 OD dose 1 per dose Treatment for 6 months Total cost = 1 x 180 x 3 =540 to save one life WOULD YOU PRESCRIBE IT?

Causes DVT in 30% !!

Any Questions?