Decision Support Systems

Decision Support Systems
Ida Sim, MD, PhD March 1, 2011 Division of General Internal Medicine, and the Center for Clinical and Translational Informatics UCSF
Copyright Ida Sim, 2011. All federal and state rights reserved for all original material presented in this course through any medium, including lecture or print.
March 1, 2011: I. Sim
Decision Support Systems Medical Informatics Epi 206
Outline
Decision support systems
background, definition
How decision support systems think

rule-based systems neural networks
CDSS Effectiveness CDSS Adoption CDSS in the Age of Watson

Big Picture of Health Informatics

PATIENT CARE / WELLNES RESEARCH
Decision support
Medical logic
Clinical Decision Support Systems
Virtual Patient
Medical knowledge
EHR
Transactions
Clinical research transactions
Raw data
Raw research data
Workflow modeling and support, usability, cognitive support, computer-supported cooperative work (CSCW), etc. 3
What is a Decision? Logic?

An action that consumes resources in the real world Logic
Oxford English Dictionary
reasoning conducted or assessed according to strict principles of validity
Merriam Webster
interrelation or sequence of facts or events when seen as inevitable or predictable something that forces a decision apart from or in opposition to reason
Decision or Logic?
Decision Logic
Diabetics with hypertension should be started on ACEI, ARB, or other
Decision or Logic?
Decision Logic X
Diabetics with hypertension should be started on ACEI, ARB, or other I prescribe lisinopril for Mrs. Chan (diabetic, BP 156/92)
Decision or Logic?
Decision Logic X
I prescribe amlodipine for Mrs. Chan (diabetic, BP 156/92)
Decision or Logic?
Decision Logic X
I prescribe amlodipine for Mrs. Chan (diabetic, BP 156/92)
X X
Clinical Decision Support

Clinical decision support system (CDSS)
software that is designed to be a direct aid to clinical decisionmaking
in which the characteristics of an individual patient are matched to a computerized clinical knowledge base
and patient-specific assessments or recommendations are then presented to the clinician and/or the patient for a decision
(Sim et al, JAMIA, 2001)
Examples of clinical decisions to be supported?
Major Target Tasks of CDSSs

Diagnostic support
DxPlain, QMR
Drug dosing
aminoglycoside, theophylline, warfarin
Preventive care
reminders for vaccinations, mammograms
Disease management
diabetes, hypertension, AIDS, asthma
Test ordering, drug prescription

reducing daily CBCs in hospital, drug allergy checking
Utilization
referral management, clinic followup
What Isnt a CDSS

Medline UpToDate Static guideline repositories
www.guideline.gov (National Guideline Clearinghouse)
Online laboratory data, test results, chart notes Retrospective quality improvement reports
how your vaccination rates compare to your colleagues
A CDSS?
Chief complaint: Symptoms for $400 please Symptom: Chest pain and shortness of breath
Dr. Watson: What is pulmonary embolism!
Outline


Basic Decision Support Task

Decision support
given starting conditions and a defined set of action choices, recommend or rank action choices for user
Requires some thinking to recommend or rank

strictly deterministic thinking thinking with fuzziness and probabilistic features
in the starting data or the reasoning procedure in the outcomes (e.g. prob. of adverse reaction)
often involves thinking about concepts (e.g., abnormal) as well as numbers

symbolic vs. quantitative computing
Decision Support Thinking

Strictly deterministic, e.g.,
first-order logic rule-based systems adhoc rule-based systems (non-mathemetical reasoning about probability)
e.g., if high WBC AND cough AND fever AND abn. CXR then likelihood of pneumonia is 4 out of 5
Probabilistic/fuzzy, e.g.,
bayesian networks
formal probabilistic reasoning, extension of decision analysis
neural networks fuzzy logic, genetic algorithms, case-based reasoning, etc., or hybrids of these
Forward Chaining Rules

Forward chaining/reasoning (data-driven)
start with data, execute applicable rules, see if new conclusions trigger other rules, and so on example
if HIGH-WBC and COUGH and FEVER and ABN-CXR => PNEUMONIA if PNEUMONIA => GIVE-ANTIBIOTICS if GIVE-ANTIBIOTICS => CHECK-ALLERGIES if PNEUMONIA and GIVE-ANTIBIOTICS and NOT (ALLERGIC-DOXYCYCLINE) => GIVE-DOXYCYCLINE
use if sparse data

Backward Chaining Rules

Backward chaining/reasoning (goal-driven)
start with goal rule, determine whether goal rule is true by evaluating the truth of each necessary premise example
patient with lots of findings and symptoms is this lupus? => are 4 or more ACR criteria satisfied?
malar rash? discoid rash? skin photosensitivity? etc
if 4 or more ACR criteria true => systemic lupus
use if lots of data

Rule Reasoning Problems

Combinatorial explosion of rules
need rule for each contingency
if MOD-WBC and COUGH and FEVER and ABN-CXR => PNEUMONIA
Rules may be contradictory

if COUGH and ABN-CXR => INTERSTITIAL-LUNG-DZ
Rules may be circular
Representational Challenges
Need to use standard vocabulary terms
need to manage evolution of vocabularies (e.g., changing terminologies in psychiatry: no Aspergers in new DSM-V)
Rules may involve complex semantic relationships

if NEPHROPATHY caused-by DIABETES
caused solely by? predominantly by? what if Im not sure? 20% sure? 80% sure?
if SINUSITIS greater than 6 months

representing temporal relationships requires 2nd order logic
Need knowledge engineering and clinical expertise to build and maintain the knowledge base over time
need to keep rules up-to-date with latest evidence
Sharing Rules
Why not have libraries of rules? Reusable, central upkeep, evidence-based... AHRQ funding library of e-recommendations
see structured logic statement for mammography screening
Morningside initiative1
public private initiative to define organizational and technical approach to sharing CDS rules
includes VA, Kaiser, DoD, AMIA, Partners, Intermountain, ASU, etc.
1http://www.tatrc.org/docs/2010-8-6-Morningside-Article.pdf
Summary of Rule-Based Systems

Deterministic, relatively simple reasoning Combinatorial explosion even for small domains Requires extensive knowledge engineering and clinical expertise Rules are difficult to share But remain most widely used method due to simplicity for small problems
Outline


Neural Networks
Finds a non-linear relationship between input parameters and output state Structure of network
usually input, output, and 1-2 hidden fully connected layers
each connection has a weight
NN for MI Diagnosis
Inputs (e.g., all patient characteristics in the EHR)
EKG findings (ST elevation, old Qs) rales (Yes, No) JVD (in cm)
Outputs are the set of possible outcomes/diagnoses

EKG findings Rales JVD Response to TNG
Acute MI No Acute MI
Training the Neural Network

Network gets trained
give examples of known patients and diagnoses
can handle missing data
system iteratively adjusts connection weights to find the network pattern that associates sets of input variables (patients) with right output state (MI or not)
Test accuracy on another set of patients In Baxts MI neural network

training set: 130 pts with MI, 120 without test set: 1070 UCSD ER patients with anterior chest pain
Baxts Acute MI Neural Net

Evaluation results: prevalence of MI 7% (Lancet, 1996)
Sensitivit y Physicians Neural N et Specifi city 73.3% (63.3-83.3) 81.1% (78.7 83.5) 96.0% (91.2 100) 96.0% (94.8 97.2)
Results were driven by non-standard predictors

rales, jugular venous distention
Why wasnt this neural network used more widely?

black box nature limits explanatory ability and lessens acceptance users have to input the variables manually
interfacing to EHRs would increase adoption need to define and code rales and other input terms
Outline


CPOE and Medication Safety

1998: CPOE reduced medication errors 55%1 2005: Qualitative study found 22 error types promoted by CPOE, quite common2 2008: Systematic review of 10/543 citations, no RCTs3
5 studies (P <= .05) for ADE reduction, 5 n.s.
2011: CPOE part of Meaningful Use criteria

more than 30% of patients with at least one medication on their medication list have at least one medication ordered through CPOE
1Bates
JAMA 1998;280:1311-1316. JAMA. 2005 Mar 9;293(10):1197-203 3Wolfstadt J Gen Intern Med. 2008 Apr;23(4):451-8.
2Koppel
Is Decision Support Effective?

2005 systematic review of CDSS effectiveness1
diagnosis: 4/10 (40%) studies beneficial reminder systems: 16/21 (76%) disease management systems: 23/37 (62%) drug dosing: 19/29 (66%) few studies improved patient outcomes: 7/52 (13%)
Counted the number of systems in each category that were effective (p>0.05)
but CDSS not all the same! (apples and oranges)
1Garg
et al. JAMA 2005 293(10):1223-1238
CDSS Running Example

Hypertension treatment Clinical Decision Support System (CDSS)
Clinic has an EHR During patient visit, CDSS notes that BP and trend is too high CDSS checks patients Cr, diabetes status, cardiac status, current meds and allergies and recommends drug therapy change according to JNC VII guidelines and insurance coverage Presents e-prescription for MD to verify. If verified, order is sent directly to pharmacy and medication list updated
Apples HTN CDSS

Clinical Decision Support Systems (CDSSs)
software designed to directly aid clinical decision-making
helps clinician to prescribe anti-hypertensive
in which the characteristics of an individual patient are matched to a computerized knowledge base
matches EHR and other data to computable guideline
and patient-specific assessments or recommendations are presented to the clinician and/or patient for a decision
recommends drug according to clinical, guideline, and insurance information provides clinician with decision choice to prescribe or not prescribe
Oranges HTN CDSS

Clinical Decision Support Systems (CDSSs)
software designed to directly aid clinical decision-making
help clinician to prescribe anti-hypertensive
in which the characteristics of an individual patient are matched to a computerized knowledge base
clerk routinely abstracts current BP, A1C, meds, allergies and insurance status from paper chart into a database computer runs pt information against computerized guideline computer outputs a piece of paper with recommendation
and patient-specific assessments or recommendations are presented to the clinician and/or patient for a decision
MD given piece of paper with individualized drug recommendation MD writes prescription in usual paper-based way
Taxonomy of CDSSs
KNOWLEDGE/DATA SOURCE Clinical knowledge source [ ] Patient data source [ ] Data source intermediary [ ] Degree of customization Update mechanism
DECISION SUPPORT Reasoning method Clinical urgency Recommendation explicitness Logistical complexity Response requirement
CONTEXT Target decision maker Clinical setting Clinical task Unit of optimization Relation to point of care Potential external barriers to action WORKFLOW Degree of workflow integration
INFORMATION DELIVERY Delivery format Delivery mode Action integration Delivery interactivity/explanation availability
OR
System user/ Target decision maker
System user/Output intermediary [ ]
Target decision maker
CDSS Characteristics
Using taxonomy, reviewed and classified 42 RCTevaluated CDSSs Tremendous variation in decision-maker/context, how recommendation delivered, staff needed to make system run, complexity of recommended actions
45% targeted to clinician, 55% patient, 5% both 62% based on national guidelines or literature 69% pushed recommendation to decision maker 43% collected data directly from the EHR
45% required data input intermediary (11% MD)
26% required an output intermediary
Generalizing successes from literature is difficult

(Berlin, Sim, 2006)
CDSS Effectiveness Summary

Systematic review of systematic reviews on Impact of eHealth on Quality & Safety
many of the clinical claims made about the most commonly deployed eHealth technologies cannot be substantiated by the empirical evidence.1
Findings limited by
methodological problems and design type of studies insufficient appreciation of workflow component of CDSSs insufficient appreciation of heterogeneity of systems
1Black
et al, PLoS Med 2011 8(1):e1000387
Outline

rule-based systems
neural networks

Low CDSS Adoption

Adoption of CDSSs beyond simple reminders
< 10% of those with EHRs
Reasons
informatics
technical
organizational / financial fundamental conundrum
Informatics Barriers
Requires computation across Data, Information, Knowledge
data is often qualitative, fuzzy
how to represent looks sick, severe pneumonia
information (meta-data) often not easily available

e.g., seen in another ER last week for same problem
lots of tacit vs. explicit knowledge required
Most CDSSs are rule-based systems

combinatorial explosion, rules not shared, updated... inability to handle probabilistic outcomes, values
Computer best at data-intensive simplistic deterministic decisions (augmenting intelligence) vs. knowledgeintensive, probabilistic, value-based decisions
Technical Barriers
CDSS has to interface to local data systems
manual double-entry input is a no-go Meaningful Use establishes CCD1 as standard EHR exchange format
e.g., Problem List, Allergies, K+ value new industry-wide green CCD standard due May 2011
Exchange standard may not be granular enough for CDSS

e.g., Allergies as free text, vs. med and reaction
Need standardized (i.e., coded) input data

e.g., whats in the Past Medical History field?
1http://www.hl7.org/implement/standards/cda.cfm
Organizational Barriers
CDSSs are complex workflow interventions
high requirement for complementary innovations
requires organizational change leadership and expertise
Lack of or uncertain incentives/rewards for better quality

e.g., accountable care organization rules imminent (Berwick D, CMS Administrator, Feb 24, 2011)
Outline


A Huge Advance in Computing

Watson winning Jeopardy! is like Deep Blue beating chess grandmaster Gary Kasparov (1997)
but Deep Blue less successful than expected at solving protein folding problems
First industry that IBM wants to apply Watson to is health care,1 partnering with
Nuance voice recognition company Columbia University on clinical decision support U. Maryland on imaging
1http://www-03.ibm.com/press/us/en/pressrelease/33726.wss
AI Background
Artificial intelligence holy grail
chess is a highly structured game with defined rules and solutions (just a lot of them) Jeopardy! also a narrow game (a question answering game) but played using natural language requiring processing (=NLP)
Factoid question answering is a specific kind of intelligence

"The antagonist of Stevenson's Treasure Island -- Who is Long-John Silver?, vs. What triggered the revolution in Egypt? What causes Chronic Fatigue Syndrome? vs. Book the cheapest, most convenient transportation for my family for this 4 city trip to Spain in July
Similarities/Differences
Jeopardy!
very large scope natural language full of puns, ambiguities corpus is free text only all fact based there exists one and only one right answer right answer is in the corpus somewhere, requiring only syntactic (ie grammatical) processing to get at is one shot

Clin Decision Support

very large scope clinical notes and literature highly idiosyncratic natural language corpus includes text, numbers, images (MRIs), video (eg echo) not only facts (should pt. be on warfarin to prevent stroke?) often no single right answer, best answer requires semantic (I.e., meaning) processing
e.g., azithromycin, critical appraisal of literature
often requires back and forth (e.g., to clarify context, values, constraints)
Watson in the Big Picture

PATIENT CARE / WELLNES
Watson
RESEARCH
Virtual Patient Decision support Medical logic
Medical knowledge
Transactions
Clinical research transactions
Raw data
Raw research data Nuance voice recognition
Workflow modeling and support, usability, cognitive support, computer-supported cooperative work (CSCW), etc.
The Jeopardy!/Medical Logic Problem

Voice recognition (picking out words from speech)
Watson: was given questions in written text Clinical: Dragon Dictate etc moderately good for restricted domains (e.g., radiology)
Understand the sentence/question

Watson: The antagonist of Stevensons Treasure Island Clinical: What antibiotics treat pertussis?
Go look for candidate answers in the corpus of knowledge

Watson: free text Project Gutenberg, wikipedia, dictionaries, encyclopedias, newspaper articles, etc. Clinical: EHR, PubMed, UpToDate, all of Internet? free text, images, numbers, video, data streams (eg GPS, ICU data)
Score answers for likely correctness Give best answer (or rank answers and be able to explain why)
Example Jeopardy! Process

http://blog.reddit.com/2011/02/ibm-watson-researchteam-answers-your.html
Data + Watson
Computer: Ms. Lee has had paroxysmal cough for 2 weeks, with emesis.
..
Adult pertussis is a strong possibility.
Doc
Symptom inquiry, diagnosis using neural network or rule-based system
What is the current incidence of pertussis?

..
17.8 cases / 100,000 in S.F. county Jan thru December 2010
Doc
Question answering: public health reports, data, culture results, etc.
Data + Watson
Your patient is 4 months post-partum. I suggest treating presumptively for pertussis.*
Rule-based checking of EHR
I agree. Dont macrolides treat pertussis?

..
Yes, erytho, clarithro and azithromycin are the preferred antibiotics. Bactrim is second-line.
Doc
Question answering: reference sources, literature
Data + Watson
I would suggest azithromycin 500 mg on Day 1 and 250 mg on Days 2 to 5.*
*CDC guidelines 2005, local cultures uniformly sensitive to azithro, pt not allergic, azithro covered by insurance
Question answering and rulebased checking of allergies, insurance, local sensitivities
Make it so!
..
Doc
CPOE
APEX
Some Clinical AI Systems

Question Answering
askHermes http://www.askhermes.org/
Diagnostic support
http://dxplain.org/dxp/dxp.pl
http://www.isabelhealthcare.com/home/default
Fundamental CDSS Conundrum

Better quality care <-- better decision support Better decision support <-- smarter systems
know more about the patient, evidence, context
Smarter systems <-- more richly coded D-I-K

for EHR: SNOMED, standard EMR structure for knowledge: coding, structures for guidelines, RCTs
Coded data <-- Coding of data entry Coding of data entry <-- Greater physician time Greater physician time --> no play --> no gain
Implications
Clear trade-off between physician coding effort and smarter decision support
can NLP help? do we really need to code if we have Watson-like abilities to understand (un)natural language
For now, dont expect more decision support than coding allows generally successful decision support
preventive care: age, last mammogram, etc. allergies: Yes/No on specific drugs drug dosing: weight, height, creatinine, age
generally unsuccessful decision support

diagnostic assistance complicated therapies (e.g., management of hypertension, treatment of depression)
Summary on Decision Support

Most CDSSs are rule-based Equivocal evidence of benefit
workflow/organizational inputs underappreciated
Fundamental trade-off between

effort of coding data and quality of decision support
Greater decision support adoption will require

wider EHR use, better interoperability, more coding or far more powerful NLP
Need to be realistic on what decisions computers can best support


Decision Support Systems

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Decision Support Systems

Uploaded by

Copyright:

Available Formats

Decision Support Systems

March 1, 2011: I. Sim