Professional Documents
Culture Documents
ADESSA AMIA Presentation Jon Duke
ADESSA AMIA Presentation Jon Duke
ADESSA AMIA Presentation Jon Duke
A Real-Time Decision Support Service for Delivery of Seman:cally Coded Adverse Drug Event Data
Overview
Introduc@on
to
ADESSA
Concept
and
Design
Evalua@on
Study
Results
Discussion
/
Future
Work
ADESSA is a web service designed to provide labeled adverse drug event data in coded, structured form
Origins of ADESSA
The Problem
SIDER
CO-START
Terminology
Limited
drug
set
Infrequently
updated
Op@mal
Solu@on
Comprehensive
source
of
labled
ADE
data
MedDRA
coded
useful
for
pharmacovigilance
/
ADE
repor@ng
natural
phrasings
of
adverse
events,
human-readable
SNOMED-CT
coded
useful
for
integra@on
with
clinical
decision
support
Web service capable of retrieving data for one or more drugs using standard medica@on iden@ers
System
Components
Structured
Product
Label
Informa@on
Coder
and
ExtractoR
(SPLICER)
Quan@ta@ve
Adverse
Reac@on
Knowledgebase
(QuARK)
Adverse
Drug
Event
Support
Service
(ADESSA)
SPL Parser
Section Extraction XML tag removal Text Normalization Sentence Detection
ADE Extractor
Processors List MedDRA term identifier Table
MedDRA Mapping
Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper
ADE Extractor
Processors List MedDRA term identifier Table
MedDRA Mapping
Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper
SPL Parser
Section Identification XML tag removal Text Normalization Sentence Detection
ADE Extractor
Processors List MedDRA term identifier Table
MedDRA Mapping
Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper
SPL Parser
Section Extraction XML tag removal Text Normalization Sentence Detection
ADE Extractor Extracts
raw
ADEs
List from
clean
SPL
Table
Processors MedDRA term sweeper
MedDRA Mapping
Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper
SPL Parser
Section Extraction XML tag removal Text Normalization Sentence Detection
ADE Extractor
Processors List MedDRA term identifier Table
MedDRA Mapping Processors ExactMatch Synonym StopStem Transform Similarity Tokenizer LaboratoryMapper
Database
Crea@on
Retrieved
5782
SPLs
from
DailyMed
Used
SPLICER
to
extract
and
map
the
ADEs
from
these
labels
to
MedDRA
preferred
terms
Linked
SPLs
to
RxCUIs
using
RxNorm
via
SPL_SET_ID
via
NDC
codes
Database
Crea@on
Successfully
processed
5602
SPLs
Extracted
545
602
ADEs
Per
Label
180
not
processed
due
to
XML
varia@on
1706
generic
medica@ons
22
097
unique
raw
phrases
8709
unique
MedDRA
terms
Mean
ADEs:
70
Median
ADEs:
49
Range:
0
to
525
ADESSA
Overview
CCD
Level
3
PHP/MySQL
NuSOAP
Labeled ADEs
RxCUI
MedDRA SNOMED-CT
QuARK
Evalua@on
Goals
Assess
the
accuracy
of
the
adverse
event
extrac@on
process
Assess
the
comprehensiveness
and
performance
of
the
service
using
real-world
data
NLP
Evalua@on
Independent
manual
review
of
SPLICER
output
by
a
MedDRA
specialist
For
100
random
labels,
reviewer
marked
ADEs
True
Posi:ve
ADE
in
both
output
and
label
False
Posi:ve
ADE
claimed
by
SPLICER
but
either
not
present
in
label
or
not
a
true
adverse
event
False
Nega:ve
ADE
present
in
label
but
not
found
by
SPLICER
or
not
correctly
mapped
NLP
Evalua@on
Out
of
9529
ADEs:
9064
True
Posi@ves
465
False
Posi@ves
706
False
Nega@ves
Recall:
92.8%
Precision:
95.1%
F-Measure:
0.94
Results:
DSS
For
30
CCDs,
drug
count
ranged
4-37,
mean
12.2
ADEs
successfully
returned
for
157/181
(88%)
of
these
drugs
Mean
response
@me
620ms
Reasons
for
failure
to
return
results
(n=24)
7
due
to
CCDs
without
RxCUI
codes
17
due
to
incomplete
RxCUI
to
SPL
mappings
in
QuARK
In
no
case
was
the
target
drug
missing
from
the
database
Discussion
SPLICER
extracted
labeled
adverse
event
data
with
high
precision
and
recall
ADESSA
returned
pa@ent-specic
ADE
data
rapidly
upon
receipt
of
a
CCD
/
RxNorm
codes
Poten@al
value
in:
adverse
event
repor@ng
(checking
against
known
adverse
reac@ons)
clinical
decision
support
(associa@ng
pa@ent
condi@ons
with
poten@ally
causa@ve
agents)
Limita@ons
Even
with
95%
precision,
s@ll
~25,000
incorrect
asser@ons
in
database
Sta@c
set
does
not
account
for
new
and
updated
labels
Encountered
some
issues
with
mapping
of
RxCUIs
to
SPLs
SNOMED
concepts
may
not
reect
original
nuanced
ADE
concept
Conclusions
Extrac@on
via
NLP
and
dissemina@on
via
DSS
can
provide
coded
ADE
data
for
use
in
CDS
Future
work
includes
Search
by
specic
ADE
or
ADE
category
Support
for
real-@me
updates
of
QuARK
using
FDAs
RSS
feed
for
new
/
updated
labels
Broadening
of
ADESSA
for
use
outside
rewall
Thanks!
Ques@ons?
Jon
Duke
jduke@regenstrief.org