Literature Evaluation

Definitions
 Systematic approach – A series of seven steps that promotes effective and efficient responses to drug
information requests
o When searching drug information literature, it is best to start with tertiary resources and then
move on to secondary and primary resources
 Tertiary resources – Provides the reader with preliminary quick background and knowledge of the
topic. These resources summarize the current standard of practice.
 Secondary resources – Indexing libraries typically used when tertiary resources are incomplete
 Primary resources – Studies and original reports of data that need to be critically examined by the
reader. Provides details for how a study was conducted.

Why is the systematic approach important?

 Obtaining the correct information to answer a question enforces the Seven Patient Rights
 Systematic approach steps:
1. Secure the demographics of the requestor Used to help a pharmacist
2. Obtain background information understand the true
3. Determine and categorize the ultimate question question that is being asked
4. Develop a strategy and conduct a search
5. Perform evaluation, analysis, and synthesis
6. Formulate and provide a response
7. Conduct follow-up and documentation

How do we evaluate tertiary literature?

 Examples of tertiary literature: General textbooks, Lexicomp, Micromedex, review articles (narrative
or systematic reviews and meta-analysis), and internet sources
 Evaluation considerations: author expertise, purpose of the source, edition/year of publication,
references cited, ease of use, format, and availability

Strengths Limitations
 Easy to use  Textbooks may be out of date
 Concise  More granular information may be left out for the
 Readily accessible to most institutions sake of summarization/brevity
 Expensive if unaffiliated with an institution
How do you search secondary literature?
 Identify appropriated databases based on your drug information question
o PubMed/MEDLINE – easily accessible and commonly used
o International Pharmaceutical Abstracts – covers many pharmaceutical journals that are not
included in MEDLINE; includes abstracts from pharmaceutical conferences
o Journal Watch – NEJM product; summarizes published medicine into specified categories
o Newsletters (ex. ClinAlert, Reactions Weekly) – provide information regarding adverse drug
rections
 Search for information using a “controlled vocabulary”
o PubMed: MeSH terms, boolean operators (“AND”, “OR”, “NOT”)
 Articles indexed by MeSH terms are usually the most relevant to a given search, but
usually do not include the most recent published literature on a topic
 Apply limits to your search
o “Human”, “Within the past 5 years”, “Randomized Controlled Trial”
How do we evaluate primary literature?
 Definitions
o Bias – A systematic error that affects the result either positively or negatively; something that
affects the study results other than the treatment under investigation
o Validity – Evaluates the integrity of a study (internal validity) and how well the study can be
applied to practice (external validity)
o Confounding variables – variables related to the condition being studied that may affect the
study outcome
 Clinical trial designs:
o Proven causality trials from least to most relative strength:
 cross-sectional studies/case reports  cohort and case-control studies  systematic
reviews/meta analyses  randomized controlled trials
Study Type Study Category Definition
Parallel Interventional  Patient assigned to one treatment for a period of time
 Preferred in acute diseases or studies with curative intent
Crossover Interventional  A patient receives one treatment (standard of care), washout
period, then studied intervention
 Patient serves as their own control
Cohort Observational  Patients exposed to a factor are observed prospectively
(follow-up)  Useful in determining frequency of adverse events
Case-control Observational  Similar to cohort but retrospective; compares patients with no
exposure to a factor (control) to those exposed (cases)
 Useful for studies with low recruitment rates
Cross- Observational  Survey characteristics of a population during a specific period of
sectional time; useful for epidemiologic studies
 Prevalence vs incidence
 Correlation vs causation
Case Observational  Documented observations related to a single or group of patients
reports/series  Useful in rare diseases, adverse effects, and teratogenicity

2
 Randomized Control Trial Structure
Title  Reveals the purpose of the study, but should not allow the readers to draw any
conclusions about the results
o “Amoxicillin is superior to cephalexin for treating acute otitis media”
o “The safety and efficacy of amoxicillin and cephalexin for treating acute
otitis media”
o A comparison of two antibiotics for treating pediatric infections”
Author Affiliation  Disclosures section; reveals any conflict of interest authors may have had
 References and sponsorship
Abstract  <250 word summary usually including hypothesis, setting, objectives, methods,
results, and conclusion
 Reading the abstract is not a substitute for reading the entire article
Introduction  Identifies all relevant background information as well as the study objective
 Should explain what is being tested, why it is being tested, who is being tested, and
how the tests will be conducted
 References from the introduction should be analyzed
Inclusion/Exclusion Criteria  Used to determine external validity
Interventions and Controls  Details the drug dose, route, duration, and description of any placebo used
 Placebo vs active controls
 Historical controls
Outcome Measures  Need to be clearly defined before the initiation of a study
 Primary outcome vs secondary outcomes
Randomization  Simple vs restricted randomization
Blinding  Single-blinded – Either the investigator OR the patient is blinded to treatment
 Double-blinded – BOTH investigator and patient are blinded to treatment
 Triple-blinded – Investigator, patient, AND group who analyzes the data are
unaware of treatment groups
 PROBE – Prospective randomized observational endpoint-blinded trial
Statistics Be aware of different types of errors (I and II) and statistical vs clinical significance
Patient populations  Sample size: determined through a power analysis (α, β, expected difference
between groups, and variation/standard deviation)
 Evaluates if there were enough patients in the study to detect a difference
 Will compensate for expected dropouts
 Intention to treat vs per protocol
 Most common method to account for patients who dropped out of the study is last
observation carried forward
Interim Analysis  Occurs when the investigator evaluates the data early with a specific date cutoff
Result Analysis Discussion
 Baseline Characteristics  Summary of important results
o Give a detailed picture of patient population  Explanation of the mechanism behind the results
studied (who was invited to the party vs who  Comparison of the results to other published
came) studies
o Can compare groups to see if intervention  Limitations of the study
and control groups had similar characteristics  Discussion of the clinical implications and
 Flow Diagram generalizability of the results
o Monitor dropouts in study during each phase  Biased language
of treatment o “clearly superior”, “trends toward
o Enrollment vs final analysis significant” “highly significant”
Data and 95% Confidence Intervals

3
The Publication Process

 Peer review
o Review of manuscript by a panel of experts selected by the editor
o Identify the appropriate use of study design and methods, adequate description of hypothesis and
methods, thorough data analysis, appropriate conclusions based on the reported results, ethics of the
study
o Reviewer will make a recommendation of whether or not the manuscript is acceptable for publication
 Reasons for revision
o The expert panel may initially reject (or conditionally accept) a manuscript; the authors need to justify
writing choices and augment the manuscript based on the panel’s suggestion
o Editor will make the final decision for approval
o If the manuscript is totally rejected the authors may seek publication in another journal
The Internet
 The main limitation of the internet is the variable quality of a large amount of available information
 Biases and number of “hits” are how information is shown in search engines, not quality or accuracy
 Incorrect information could include misinterpretation of FDA statements, inaccurate summaries of clinical trial
results, and false claims regarding the effects of a drug
 How do we evaluate websites?
o Authorship
o Use of an advisory board
o References for the clinical content
o Disclosure of funding or sponsorship
o Timelines of the information
o Seal of approval or quality label
 The Health on the Net Foundation Code of Conduct was developed to improve the quality and
reliability of information of health care websites
 Submission is voluntary