Evidence Based Medicine Needs A Reality Check

Dr. Suresh Shottam

As doctors, we are generally encouraged to practice evidence-based medicine (EBM). To most
doctors this means medicine which has been proven to be beneficial. Doctors who take this view
however, may be shocked to learn just how little of conventional medical practice has been proven to
be beneficial.
As someone who has a solid foundation in physics and in research before getting into clinical
practice, I am compelled to point out how blind faith in EBM leads doctors to become rigid, arrogant
and narrow-minded in their views. The end result is that true progress is held back and the patient is
not optimally benefited.
But first, what exactly is Evidence Based Medicine?
Here is a definition of EBM offered by one of our colleagues in a recent online post:
EBM is the application of the scientific method to medicine. This is done by applying modern scientific
and statistical methods, to help make rational decisions for individual patients, based on the available
evidence and our existing body of knowledge. It uses the techniques from science, engineering and
statistics, such as meta-analysis of scientific literature, risk-benefit analysis, and randomized
controlled trials, it aims for the ideal that healthcare professionals should make "conscientious,
explicit, and judicious use of current best evidence" in their everyday practice.
While the scientific method started being applied to medicine as early as the 19th century,
modern EBM didn't really get its start until the advent of clinical epidemiology in 1988.
Since modern EBM got its start with the advent of clinical epidemiology in 1988, let's hear from one
of the most trusted names in epidemiology:
John P. A. Ioannidis is a Professor of Health Research and Policy at Stanford School of Medicine,
the University's Rehnborg Chair in Disease Prevention and Director of its Prevention Research
Center, and Co-Director, along with Steven Goodman, of the Meta-Research Innovation Center at
Stanford.
He was chairman at the Department of Hygiene and Epidemiology, University of Ioannina School of
Medicine as well as Adjunct Professor at Tufts University School of Medicine. He is best known for
his research and published papers on scientific studies, particularly the 2005 paper "Why Most
Published Research Findings Are False").
John Ioannidis is one of the world's most important experts on the credibility of medical research.
He and his team of researchers have repeatedly shown that many of the conclusions biomedical
researchers arrive at in their published studies are exaggerated or flat-out wrong.
Even the Most Widely Cited Data is Often Wrong
Back in 2005 Dr. John Ioannidis showed that there is less than a 50 percent chance that the results of
any randomly chosen scientific paper will be true, and there is a seemingly endless number of
variables that can affect the outcome. As Dr. Ioannidis wrote:

"In this framework, a research finding is less likely to be true when the studies conducted in a field are
smaller; when effect sizes are smaller; when there is a greater number and lesser pre-selection of
tested relationships;
where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when
there is greater financial and other interest and prejudice; and when more teams are involved in a
scientific field in chase of statistical significance.
Simulations show that for most study designs and settings, it is more likely for a research claim to be
false than true. Moreover, for many current scientific fields, claimed research findings may often be
simply accurate measures of the prevailing bias."
Again in 2008, Dr. Ioannidis showed that much scientific research being published is highly
questionable. According to that analysis, the studies most likely to be published are those
that oversell dramatic or otherwise considered important results -- results that often turn out to be
false later on.
Prestigious journals boast that they are very selective, turning down the vast majority of papers that
are submitted to them. The assumption is that they therefore publish only the best scientific work.
But Dr. Ioannidis' study of nearly 50 papers in leading journals, which had been cited by more than
1,000 other scientists showed that within only a few years, almost a third of the papers had been
refuted by other studies.
Making matters worse, the "hotter" the field, the greater the competition, and the more likely that
published research in top journals could be wrong. As David H. Freedman wrote in The Atlantic:
" Even for medicine's most influential studies, the evidence sometimes remains surprisingly
narrow. Of those 45 super-cited studies that Ioannidis focused on, 11 had never been retested.
Perhaps worse, Ioannidis found that even when a research error is exposed, it typically persists for
years or even decades.
He looked at three prominent health studies from the 1980s and 1990s that were each later soundly
refuted, and discovered that researchers continued to cite the original results as correct more often
than as flawedin one case for at least 12 years after the results were discredited."
Defenders of EBM are so taken up with the idealistic basis of EBM that they are blind to the realities
of how the scientific method has been distorted and manipulated by modern medical research to suit
hidden agendas and bias. This is a dangerous situation because doctors are lulled into a false sense
of security thinking they are practicing scientifically proven medicine when in fact it is far from the
truth.
I would like to direct the reader's attention to two important points surrounding EBM:
1. Much of modern medical practice lacks evidence (contrary to what doctors are led to believe)
2. In the few areas where there is evidence, the evidence is not being followed...and in many cases
the opposite of what the evidence points to is being practiced
Let's do a reality check on EBM then.

Proof that much of modern medical practice lacks evidence

According to the US Government's Office of Technology Assessment (US OTA) -- only 10% to 20%
of all procedures currently used in medical practice are supported by controlled clinical studies.
The British Medical Journal's "Clinical Evidence" analyzed common medical treatments to evaluate
which are supported by sufficient reliable evidence (BMJ, 2007). They reviewed approximately 2,500
treatments and found:
- 13 % were found to be beneficial
- 23 % were likely to be beneficial
- 8 % were as likely to be harmful as beneficial
- 6 % were unlikely to be beneficial
- 4 % were likely to be harmful or ineffective.
- 46 % were unknown whether they were efficacious or harmful
(Remember, the above findings are based on 2,500 COMMONLY USED MEDICAL TREATMENTS).
Published Results In Medical Journals Often Don't Match Research Findings
Dr. Beatrice Golomb, MD, PhD, explains that there is a qualitative difference between the scientific
facts gathered from a study and the conclusions drawn.
She said, "The FDA conducted an analysis of antidepressant drug trials and found that of 38 trials for
which the evidence appeared favorable, 37 had been published. Whereas of 36 trials for which the
evidence appeared unfavorable, 22 were not published at all, and 11 were published in a way that
misleadingly conveyed the outcome as though it was favorable.
The published papers led the reader to believe that over 90 percent of the published studies were
favorable, when in truth that figure was only about 50 percent.
So that's an example of how the evidence we are led to believe is dramatically different from the
evidence that the actual study produced."
In other words, drug makers do an excellent job of publishing the findings they want you to
know, while hiding negative findings.
In the few areas where there is evidence, the evidence is not being followed
Rhinosinusitis and Antibiotics
"Almost half of patients with upper respiratory tract infections in the U.S. still receive antibiotics from
their doctor" even though "the CDC warns that 90% of upper respiratory infections, including
children's ear infections, are viral, and antibiotics don't treat viral infection. More than 40% of about 50
million prescriptions for antibiotics each year in physicians' offices were inappropriate."

Unfortunately, although antibiotics for sinus infections may be medical gospel, the treatment falls
under the 80% of medical procedures that had never been testeduntil now. But now, it has been
tested and the verdict isn't good.
According to a study released recently in the Journal of the American Medical Association, the
accepted gospel turns out to be unacceptable. In their study of 166 adults with sinus infections, the
researchers found that those who took the antibiotic amoxicillin didn't feel better any faster than those
who received a placebo. People in both groups experienced about the same amount of relief after
three days.
To quote the report, "There is now a considerable body of evidence ... that antibiotics provide little if
any benefit for patients with clinically diagnosed acute rhinosinusitis."
Comparision of Invavsive vs. Non-Invasive Therapies
There are 39 major studies comparing bypass surgery or angioplasty with conservative medical
treatment using only drugs. Many of the studies have what is called a selection bias (i.e. the patients
were selected in such a way so as to favor surgical intervention).
In spite of these selection biases heavily favoring surgical intervention, almost every single study
clearly and unequivocally demonstrates that invasive treatment, be it bypass surgery or angioplasty,
fails to reduce heart attacks and mortality when compared to patients who have been conservatively
treated with medication. In addition, there is a clear increase in mortality, heart attack rate,
cardiovascular events, repeat angioplasty and bypass surgery in the invasively treated patients.
The above evidence clearly points to drug therapy over angioplasty and bypass surgery in a majority
of patients. But that is not the case.
With the advent of angioplasty and stents, the non-interventional cardiologist is an endangered
species. Cath Labs, Angioplasties and CABGs are very much in vogue. According to the above
studies, in many cases, these invasive procedures make more dollars than sense in a large number
of cases!
Angioplasty
In 2007, a study in the New England Journal of Medicine found that angioplasty did not save lives or
prevent heart attacks in non-emergency heart patients. In fact, according to the study, angioplasty
gave only slight and temporary relief from chest pain, the main reason it is performed -- virtually no
different than that seen with medication. Even more significant was the fact that the mortality curves
for the angioplasty and medicine groups were virtually identical during the initial 4.6 years of the
study.
Why are these results important? Because at the time the study was released, doctors were
performing about 1.2 million angioplasties a year in the United States alone, at a negotiated cost of
about $25 billion a year. That's $25 billion wasted every year in just the US -- and on just this one
medical procedure.
Since that initial study, two more studies have come to virtually the same conclusion -- that using
angioplasty to open blocked arteries to treat chest pain or angina may be riskier and no more
beneficial than medication.

Since multiple scientific studies published in major peer reviewed journals have demonstrated
repeatedly that angioplasty is a wasteful and ineffective procedure, and since medicine is evidence
based, the procedure has obviously been eliminated from the medical repertoire, right?
Wrong!
In all, the reduction in angioplasty surgery since the evidence of its ineffectiveness was first published
amounts to a mere 10-15% drop from its peak. Even more startling is that upon being presented with
the clear evidence of angioplasty's ineffectiveness, the response of the medical community was not to
decrease the number of angioplasties performed in the US -- but to decrease the amount of time
required to get one once you're admitted to the hospital. As the American Heart Association said with
great pride just a few days ago, "More than 90 percent of heart attack patients who require an
emergency artery-opening procedure known as angioplasty are treated within the recommended 90
minutes, compared to less than half five years before."
The Truth About Statins
However, 5 major statin drug trials (PROSPER, ALLHAT-LLT, ASCOT-LLA, AFCAPS and
WOSCOPS) show that statin drugs provided a ridiculously low Absolute Risk Reduction in total
mortality of 0.3%...while guaranteeing a host of side-effects such as rhabdomyolysis.
Avastin - Going Against The Evidence
A report out of Stony Brook University in New York confirmed that the cancer drug Avastin causes
potentially lethal holes in the colon -- a most disturbing side effect for a drug often prescribed for
advanced colon cancer. Sales of Avastin run about $4.4 billion a year.
The Greatly Over-hyped FDA Approval of Drugs
To get FDA approval to market a drug, most of the studies for psychiatric conditions last only six
weeks. In view of the fact that most people take anti-depressant or anti-anxiety medicines for many
years, how can anyone consider these short-term studies scientifically valid?
What is so little known and so sobering is that research to date has found that placebos were 80
percent as effective as the drugs-with fewer side effects.
- Marcia Angell MD, Ex-Editor, The New England Journal of Medicine
Conclusion
All of this is certainly not EBM.
When I speak about this issue, many doctors immediately rush to defend EBM by saying that it's not
perfect, but it's the best system we've got.
That is not an acceptable argument because it is fatally flawed in its logic.
Firstly, perfection is not required in order to have a workable system.
Secondly, it is not the best system we've got...here's why...

There's a Problem With "Scientific" Experiments such as Randomized, Double Blind, PlaceboControlled Testing
"Scientific" testing is based on the belief that the treatment group differs from the control group only in
the factor being tested. While this works well in testing inanimate objects (as in the physical
sciences), it is far from clear that such testing works all that well in health matters. Living organisms
are simply too complex and illness is often so multi-factorial that parity of the treatment group and the
control group cannot be presumed.
In the case of hormones and osteoporosis, operative factors include sex hormone binding globulin
(SHBG) and cortisol binding protein (CBG), genetic traits, exercise, sun exposure, digestive
differences, stress, program compliance, adiposity, diet, nutrients such as magnesium, calcium,
silicon, and boron, or vitamins such as C, A, E, D, and K, as well as the influence of other hormones
such as androgens, growth hormone, and thyroid. Also involved are numerous drugs and medications
used by the test subjects. While some of these factors (such as height, weight, adiposity, and
medications) are easily "controlled" (i.e., accounted for), many of them are not.
Obviously, the more variables there are, the more difficult it is to assure parity between the treatment
and the control groups. Medical history is studded with examples of spurious conclusions resulting
from uncontrolled or unrecognized variable factors.
Another Problem With Scientific Studies
When a patient takes more than one drug at a time, the research conducted on each of the drugs
individually becomes meaningless.
Considering how many people take two or more drugs together raises serious doubts about the
scientific ground on which physicians stand (except in those few instances when a multiple-drug
protocol has been tested, as has occurred with some drugs in the treatment of people with AIDS).
To make matters worse, the already flawed system has been hijacked by an industry that favors
financial interests over science.
Sadly, physicians do not see that there is something fundamentally wrong with the present medical
model.
We need to create a better system. One that rises above the career interests of researchers, dodgy
tactics of pharmaceutical empires and top line focus of health insurance companies.
It's time to stop worshipping at the altar of EBM and wake up to the reality of a system gone awry.