Artificial Intelligence in

Practical Automation
Where AI currently is automating and
recognizing the road ahead
October 2019

Copyright © 2017 IQVIA. All rights reserved.

Contemplating the Impact of AI on Healthcare

“We always overestimate the change that will

occur in the next two years and underestimate
the change that will occur in the next 10.”
- Bill Gates

1
AI in Clinical Development Status
Lots of hype; Ready for Impact?

DeepMind's Losses
and the Future of AI market for healthcare is expected
Artificial Intelligence to top $35 billion by 2020

AI and Societal Impact –

Addressing Large,
Complex Unresolved
Problems with AI

AI Moves Forward,
Despite Challenges
and Pitfalls
2
Staying ahead of the hype curve
The IQVIA model

• IA rather than AI,

i.e. “consumer-grade AI”
• Focused on end-user
• Highly collaborative &
built around domain
experts
o Physicians deeply
embedded & at the
center of the decision
making process
• Highly iterative

3
 Clinical Planning & Strategy

Topics
Table of Clinical Trial Execution
Contents

Extreme Automation

4
AIML in CORE-powered Site Selection
Improves Performance

Predictive Machine Learning Algorithms

30%
FASTER
average
Quality
recruitment
• Protocol Deviations Outputs for rate
• Screen Failure Rate, Operational
SAEs & AEs Teams
• Overdue Actions
• Query Rate
IQVIA Metamorphosis Study
Human
Data
Science
•
•
Engine
Clinical Extraction
Curation
Participation
• Prior Trial Experience
& Attributes
Specific
Data
Action
Ready
Investigator
and Site
60%
FEWER
Cloud •
non-enrolling
Validation • Practice Attributes List sites
• Integration
Performance
• Prior Trial Experience
• Enrollment Measures
5
Augmented Intelligence – Predictive Enrollment
Smart recruitment velocity prediction through assessment of the variables that impact Clinical Trials

Early Strategy Planning Operational Support Augmented Intelligence Smart curation

• Utilization of proprietary
and non-proprietary
datasets
• Predictive model to
calculate the enrollment
rate based on protocol-
specific attributes
+ +20%
Confidence in portfolio
enrollment timelines
• Scenario optimization • NLP to bridge diverse • Automatically learn from Greater accuracy and
based upon elasticity to external data sources new trends in the clinical
reliability in projecting
key variables changes space
• AI cohorting to automate trial timelines and a
(countries, sites,
features (AI Cohorting) • Builds upon IQVIA better understanding of
inclusion criteria)
breadth of experience
• XG-BOOST to predict predictors that impact
enrollment • Continuous improvement trial duration will provide:
of the models
• SHAP and to explain the
predictors • Reduction of manual
work and focus on
scenarios planning
IQVIA Enrolment Prediction
• A more predictable
<2 seconds timeline
Increase in accuracy in the enrolment rate predictions Rapid iteration of scenarios
compared to market standards (based on comparison based on alterations of
of actual studies: median p/s/m) relevant parameters
6
Using the IQVIA AIML Model to Find Rare Disease Patients Faster
Idiopathic Pulmonary Fibrosis Environment True Positive vs. False Positive Model Results
6
Receiver Operating Characteristic Curve
1
100 patients diagnosed per day in US

50% mortality within 3 years2

True Positive Rate

60-80% screen failure rate3

47 active industry-sponsored trials4

1-2 patients per site average contribution5

False Positive Rate
1 NIH statistics : https://ghr.nlm.nih.gov/condition/idiopathic-pulmonary-fibrosis#statistics
2 University College London UCL Division of Medicine https://www.breathingmatters.co.uk/2013/07/stark-facts-about-pulmonary-fibrosis
3 IQVIA’s internal data systems based on IPF studies in the past 5 years 7
4, 5, 6 IQVIA’s internal analysis
In Silico Methodology
In Silico Prediction of Clinical Trial Outcomes

Inputs Methodology and Outputs

• Target Indication • Learn adverse event propensity and efficacy from
structure of interactions among baseline population
• Protocol
factors (baseline sickness) in protocol and
• Experimental Compound chemoinformatic/biomedical feature of compound
Data Used (public and internal, partial list):

• Predict adverse events and efficacy for a novel

protocol/compound combination via insights derived
from real-world data and a deep learning architectures

8
Clinical Trial Execution

9
Statistical Monitoring of Subject Data
Implementing machine learning models to identify patient outliers across several attributes

IQVIA Centralized Monitoring Use Case

• New algorithms built to
10,008 >85% 88.9% assist in the day-to-day
Records, running time 37 Accuracy in identification of outliers (small Accuracy to detect the vital monitoring of subjects on
sec. 1,230 duplicates found sample), with scale we would expect ~ 95% signs outliers effectively clinical trials
• Successful PoC’s in 3
initials areas of focus
x1 x1 x1
chosen:
x2 , x2 ,… , x2 – Use of machine
…

xN xN xN learning to identify and

duplicate subject
registrations
– To auto-identify
subjects with abnormal
Duplicate Subject Laboratory Tests Vital Signs Outlier lab test results
Detection Outlier Detection Detection – To detect any vital
signs
Using machine learning to come Propose a dissimilarity measure Identify subjects with large or
up with the best rules to to quantify the differences small variations from baseline
automatically find similar between subject records for using Boxplot, Normal
records in dataset flagging as outliers Distribution, and linear
10
regression
 Artificial Intelligence – Safety
The high volume and repetitive processes make Safety Case Reporting a good candidate to apply
automation and smart machine capabilities to increase productivity
IQVIA Safety Case Processing
800,000+
cases processed annually
Auto ingest of Adverse
Events
Using optical character
recognition to convert AE
e-mails/PDF’s digitally and
importing to Safety System
© IQVIA 2018. All rights reserved.
68,000+
Safety Regulatory Reports
Enhanced Coding
Descriptions
Enriching the safety
information by adding 3rd party
ontology information
1/3
Input data is structured
or semi-structured
Expert Narrative Production
Machine learning from huge
amounts of previous cases
allowing more accurate
resulting narratives
• Converting the majority of
adverse event reporting
media from various
formats (often paper and
some ‘dead’ image scans)
into a digitized and
standard format. Full
auditability
• Improving efficiency and
productivity
• Leveraging semantic
ontology data to enrich
the information
• Producing a new IQVIA
XML digital library for
learning
• 20 years of history to
learn from
11
Machine learning: Lab outliers
Analytics of significant value in early detection
Trend reviews
of safety signals in the young pediatric
population on the ADHD program Visit based trends showing
patterns needing review

Outlier identification
Performing multivariate
assessment of several
labs to identify outlier
patients showing trends
across more than one lab
parameter of interest.
Normalized by patient
exposure time (from
randomization date)

12
ML Model for Predicting AE Reporting

• Ability to learn from historic adverse event

Data Prep Identify Learning Predicting reporting trends across thousands of sites
- Variables Predictors Models Risk
• Predicting reporting pattern and risks based on
behaviors modelled on predictor variables

Identifying sites with high risk in AE reporting compliance

Operationalizing
model insights for
monitoring actions

13
*ML=Machine Learning AI=Artificial Intelligence
Potential Fraud Surveillance
Variability – Data variability assessments

Potential duplicate subjects

Potential duplicate visit records

Digit preference and digit round-off

Patient visits/labs on weekend/holiday dates

14
Extreme Automation

15
Sciences
Sciences when looking at the

11 full landscape of AI. A mix of

connections each building up
on the next

It’s a blend of Data, Tech, Analytics and

Knowledge that allows the evolution into
the more complex future state AI

What’s Next
• AI augmenting human activity
• Enabling more efficient logistics
• Machine learning to aid knowledge
workers
• Content will be created using AI
• Peer-to-peer networks will create
transparency
• Consumers will become accustomed
to talking with technology
• Demand for data scientists will
surpass demand for engineers
• AI will fight challenging diseases

16
 Maturity Waves
AI Doctor
AI
Science

Cognitive
Automated
Science Reasoning
3 Waves
Our research is Open Deep Learning
Source
Maturity Level
Neural Net
mapping across 3
waves of Computer
Science; Cognitive
Bots
Science and AI Robotic
Process
Science. We are Automation
tracking the maturity
of the technologies Computer Machine Natural Language
Learning Processing
and what the major Science
players are doing as
Block
well as the start-ups Chain
Neural
Network
Computational
Statistics

Past Present Future 17

IQVIA and Automating Operations

Consistent Quality Speed Integrated

Connecting AI capabilities into any system

18
 Virtual Access to patients
IQVIA Clinical Trials remotely with
support from AI
tools for ease of use
Operations Operations
Areas of the business Clinical workflow from Trial Site
Master File to support of Engagement
that can utilize AI Project Managers and
CRAs through the use of Engaging sites
Translation <2 Virtual Assistants with easy to use
Communicating key regulatory level regulatory level
information quickly and efficiently
anytime anywhere Minutes clinical trial tools

Global IT
Providing easy to understand
Regulatory
Safety Platform clinical trial processes through
the use of AI tools that are
engaging and maintain quality
The ability to collect
Handling of new and Trusted AI tool box of capabilities
any source and that can be re-used across multiple
old data sources
identify
into centralized areas of the business to a Patient
automatically any repeatable high level of quality.
knowledge base
safety event Engagement
19
 Artificial Intelligence – eTMF POC
Mass scale centralization of document management with automation capabilities to identify the quality,
the integrity, where duplications occur and checking, classifying and indexing into standard requirements.

IQVIA Electronic Trial Master File • Increase productivity,

180k+
Documents collected per month
Auto-Identify Document
Type
Using optical character
recognition identify, strip
format, images etc and tag
into structured definitions
800+
Document types able to run
through Deep Learning
Enhanced Document
Recognition and insight
Deep Learning engine that
recognizes the document, checks
its validity, classifies quality and
content and indexes accordingly
1000+
Trials run per year on
average
Augment Smart eTMF
Additional capabilities added
into eTMF solution supporting
added automation capabilities
and connected devices
reduce defects, ensure
compliance
• Linkage with eTMF wider
workflow
• PoC (12,000 documents)
showed over 80% initial
accuracy in ‘machine
learning’ classification on
103 document types –
expecting 95% plus with
additional
documents/learning
• System runs image
processing (at pixel as
well as character levels)
• Small human validation
team still required

20
What Translation can do in a minute or so…
Utilizing NMT and other automation capabilities to package a straight forward solution to users

Translates AND Formats 21

 Driving Quality and Efficiency thru Auto-Narratives
Situation Solution
• High costs and manual data entry associated with • Deploy Auto-Narrative tool, driven by natural language
large volume of PV case narrative production processing and natural language generation

Completely automated generation

- Regenerated for every case version – no manual editing
- Includes latest follow-up highlights in a separate section at the end of the auto narrative
› If follow-up attempts were unsuccessful, prints “Multiple attempts to obtain follow-up unsuccessful. Case closed.”

22
Avg time to create Narrative reduced by 75%
 IQVIA Translation Automation Supports 11+ Languages
Automation supported by human translation
All documents, regardless of
language are loaded into a For languages where NMT is not
common interface by internal yet available, documents are
users securely transferred to the
external vendor for full translation

NMT Available

Documents NMT performs translation Human Linguist Translation For NMT languages,
received by IQVIA performs review Certificate Issued Certified Translation
fed back to engine to
help it ‘learn’

NMT not yet available

130M+ 93,000+ 50%

Documents fed through Neural Global coverage
Documents transferred to Words translated annually in production
Machine Translation annually
vendor for full translation

23
Keys to Successful AI Automation
Leadership Engagement
Engaged leaders willing to be involved
through stage gate reviews
Variety of real-time Digital Data
Historical and new structured data content
provides IQVIA with the ability to train AI Agility with rigor
tools to understand, interpret and repeat Flexible understanding in moving from
human actions research to development of products in a
AI fast, efficient and agile way

Quality Experience
Human experience working with
AI scientists to develop tools Scrutiny through MVP
Keeping up with the latest tech,
identifying potentials through MVP with
Ownership & Accountability clear ROI targets
IQVIA control over source and methods
ensures non-bias re-usable quality AI
tools

24
Building long-term foundations of AI Virtual Assistants

Tools build Tools

PM CRA Site

Understand, Interpret, Action

Con- Work
eTMF
tracts flows

Natural Language Processing

Clinical Trans- Onto-

Terms lation logies

Digital Content

RPA OCR Voice

25
Appendix

26
Impact of All Foundational / Intelligent Automation
What impact can Foundation / Intelligent Automation have on PV Processes and Cost
• We have seen significant savings by automating processes in and around current safety platforms
• Limit is due to the architecture of the platform
• Full benefits come from aligning process, people and technology

% total value potential in the reduction of processing effort that could be

Case types unlocked implementing IQVIA Foundational Safety Automation

Serious Initial 40%

Serious
Follow-up 35%

Non-serious
60%
Initial
Non-serious
Follow-up 55%

Literature
40%
Initial
Literature
35%
Follow-up 27
 Artificial Intelligence – Connect Digital Documents Customer benefit?

Mass scale centralization of document management with automation capabilities to identify the quality,
the integrity, where duplications occur and checking, classifying and indexing into standard requirements.

IQVIA Electronic Trial Master File • Increase productivity,

180k+
Documents collected per month
Auto-Identify Document
Type
Using optical character
recognition identify, strip
format, images, etc. and tag
into structured definitions
800+
Document types able to run
through Deep Learning
Enhanced Document
Recognition and insight
Deep Learning engine that
recognizes the document, checks
its validity, classifies quality and
content and indexes accordingly
1000+
Trials run per year on
average
Augment Smart eTMF
Additional capabilities added
into eTMF solution supporting
added automation capabilities
and connected devices
reduce defects, ensure
compliance
• Linkage with eTMF wider
workflow
• PoC (12,000 documents)
showed over 80% initial
accuracy in ‘machine
learning’ classification on
103 document types –
expecting 95% plus with
additional documents and
machine learning
• System runs image
processing (at pixel as
well as character levels)
• Small human validation
team still required

28

© IQVIA 2018. All rights reserved.