HealthCloud and ICOS Qualification Overview

Health Cloud
including
IBM Watson Health Cloud for Life
Science Compliance (LSC)
Hendrik Wagner, Cloud Architect

Agenda
text
– What is GxP
– Typical Workloads
– IaaS offering for GxP Workloads
© IBM Corporation 2
The full benefits of cloud have only been available to life science
companies for non-regulated workloads
Life Sciences Product Development Life Cycle
Non-regulated
Non-regulated Data
Data Regulated
Regulated Data
Data
Drug Preclinical Clinical Marketing Marketed

Research Discovery Studies Trials Application Product
Cloud designs generally FDA: “The regulatory requirements do not

do not satisfy change because the technology changes,
compliance requirements infrastructure must still be qualified and software
still must be validated”
Why and how the life science industry is regulated
Patient safety and drug or device efficacy are the two paramount concerns
Good Laboratory Practice (GLP)

 Apply to studies that evaluate the safety (toxicology) of regulated
articles under laboratory conditions
 Prescribe the practices that shall be used to plan, conduct, and report
nonclinical laboratory studies
Good Clinical Practice (GCP)

 Actually a collection of regulations governing the conduct of clinical
trials, informed consent, and ethical oversight
 Prescribe the responsibilities of sponsors and clinical investigators in
the planning, conduct, and reporting of clinical trials and the protection
of human subjects
Good Manufacturing Practice (GMP)

 Apply to the manufacture, packing, labeling, and storage of regulated
products
 Prescribe the methods used in, and the facilities and controls used for,
the manufacture, processing, packaging, and holding of regulated
articles
GxP refers to Good “anything” Practices.
HIPAA is not GxP!
HIPAA
 Protecting data – data security
 E.g. Encrypting data on rest
HIPAA can be achieved by technical means.
GxP
 Traceability: the ability to reconstruct the development history of a drug or medical
device.
 Accountability: the ability to resolve who has contributed what to the development
and when.
Documentation is a critical tool for ensuring GxP adherence. It cannot be provided ‘after
the fact’ but needs to be put in place before the first environment is setup.
Life Science Industry Regulations
• The major regulatory agencies globally

(European Medicines Agency; FDA; Japan
Ministry of Health, Labour and Welfare) all
share the same expectations of life science
companies
• Three regulations that each agency has; Good
Laboratory Practice (GLP), Good Clinical
Practice (GCP), and Good Manufacturing
Practice (GMP), known collectively as the GxP
regulations
• To avoid re-validation with different procedures
around the world, the
International Council for Harmonisation of
Technical Requirements for Pharmaceuticals
for Human Use (ICH) has been founded in
1990 providing Guidelines for Quality, Safety,
Efficacy
How global health authorities view IT systems
A computerized system is a group of entities that includes people,
hardware, software, documentation such as manuals and standard
operating procedures (SOP’s), and peripheral devices
People
Hardware
Software
SOPs
Peripherals
IT systems Hardware + Software
The key regulation in the industry, GxP, has significant impacts on IT
infrastructures
Key GxP requirements:

– Infrastructure on which software applications are run must be qualified
– Software applications that perform regulated activities must be validated
Key challenges of maintaining qualified infrastructure include:

– Time and capital to provision new infrastructure
– Time and resources to qualify infrastructure components
– Incident, problem, and change management
Validation of computerized systems
Validation is a process for establishing documented evidence, which provides

a high degree of assurance, that a computerized system operates accurately
and reliably according to the system’s predefined requirements and quality
attributes
Attributable,
legible,
contemporaneous,
original, and
accurate
Predefined requirements Documented evidence
Excerpt of a System Requirements Specification
Document  Predefined Requirements
Excpert of an executed Test Script – 2 Sample Test Steps
 Documented Evidence
Computerized System Validation comprises
QUALIFICATION
VALIDATION
OPERATIONS
Qualification of the IT infrastructure
Qualification is a process to demonstrate and document the ability for the

components of an infrastructure to fulfill specified requirements. In addition, to
have enough controls in place to ensure that this ability is present through the
lifetime of the component.
 Qualification’s sole purpose is to safeguard
and guarantee reliability, security, and business
continuity of IT infrastructure
 All international regulatory authorities view IT
infrastructure qualification as required, and as
a prerequisite for validation.
Perform Infrastructure Qualification
In order to perform the infrastructure qualification verifications, two
qualification test plans are developed, the installation qualification and the
operational qualification
1
5
Perform Infrastructure Qualification –
Master Qualification Plan
The Infrastructure Master Qualification Process (MQP) includes the following
activities/deliverables:
• Develop Infrastructure Qualification Plan (QP)

• Develop System Requirements Specification (SRS)
• Develop System Configuration Specification (SCS)
• Perform Infrastructure Qualification Verification (IQ)
– Develop IQ Test Plan
– Develop IQ Test Scripts
– Execute IQ Test Scripts
• Develop Traceability Matrix
• Develop Standard Operating Procedures (SOPs)
• Develop Qualification Summary Report (QSR)
• Issue Infrastructure Commissioning Memo
– The above documents are reviewed by the COE and stored/approved in QDMS
– This process is closed by the approval of the Infrastructure Commissioning Memo
Inhibitors to Cloud Adoption in Life Science
Cloud providers might not understand

regulatory compliance needs
Lack of transparency and
reliance on commercial certifications
Cloud operation and management
software treated as a “black box” XX
Multi-tenancy
Security concerns
d
s
f
o
–r SAP
–LDocument Management Systems
if
– Emerging Cloud Based Health Services
e • Analytics
S • Cognitive
c • IoT
i
e
n
c
e
C
u
s
t © IBM Corporation 17
IBM‘s LSC Offering
• IBM Watson Health Cloud for Life Science Compliance (LSC)

– GxP compliant IaaS offering
(not to be confused with IBM Watson Health Cloud providing Watson
Services in a public cloud)
• Current Status: LA (Limited Availability)

– Small set of customers, only
• Previously developed under TechSales organization, LSC is now

being moved under IBM Watson Health Cloud development
– New Internal Project Name WHC-Q
GxP Compliance Foundation
A robust and comprehensive Quality Management System based on ICH Q10
Management Engagement
Provide management oversight, quality metrics,
and quality planning
Subject Matter Expertise Tools

Provide leadership and deep industry Deploy industry recognized tools for
expertise in the life science regulatory document, training, and quality
compliance area with newly hired IBM management to increase compliance and
resources efficiency
Processes & Procedures Training

Develop industry best practice processes and
Develop and deliver comprehensive training
procedures to be shared and used globally
curricula designed by SMEs to ensure
account personnel are properly trained
QUALIFICATION
Internal Monitoring
Provide global oversight and implement a CAPA
VALIDATION program to support account teams servicing
regulated clients
OPERATIONS
19
19 © IBM Corporation 19
An Integrated set of validated management tools is needed to
support GxP compliance
• DMS: Enables electronic workflows, client

Document documentation access, electronic signature
Mgmt System
(DMS) • QMS: Provides audit management,
deviations, and exception workflows -
based on International Conference on
Incident, Harmonization (ICH) guideline Q10
Problem, & Quality Mgmt
Change Mgmt System (Pharmaceutical Quality System)
(IPCM) (QMS)
• LMS: Learning management includes
Dedicated scheduling, course management, learning
plans, assessments, and reporting
• AET: Management of system requirements,
Automated Learning Mgmt test cases, test execution, and automated
Evidence Tool
(AET)
System traceability to support infrastructure
(LMS)
qualification and computerized system
validation projects
• IPCM: Maintains validation deliverables
(e.g. plan, requirements, test cases) and
manages client service requests
LSC – GxP Compliant IaaS Offering –
Dedicated Cloud Infrastructure
Compute Storage Compute Storage Compute Storage
OpenStack OpenStack OpenStack
Softtlayer Data Center
Customer Customer Customer

1 2 3
LSC – GxP Compliant IaaS Offering –
Dedicated Cloud Infrastructure
used in a Customer Disaster Recovery (DR) Scenario
Application Level
Compute Storage Load Balancing and/or Compute Storage
Data Replication
OpenStack OpenStack
Softtlayer Softtlayer
DAL Data Center LON Data Center
Customer
IBM provides end to end GxP compliance hypervisor and
below saving customers massive efforts …….
Well Defined Responsibility

Customers reduce their offering setup efforts
VM 1 VM 2
from months to hours
App n
App 3
App 2
App 4
App 1
Customer
Setup Efforts Recurring Efforts

Operating Operating
GxP: IQ / OQ / Problem /
GxP IQ/OQ & Validation Incident / Change
System 1 System 2
Hypervisor
IBM
Infrastructure Setup Efforts Recurring Efforts

GxP: IQ / OQ/ Problem /
GxP IQ/OQ & Validation
Incident / Change
On infrastructure level IQ, Incident Change and Problem management, Document Management
Giving an access to the infrastructure in minutes instead of hundreds of hours enabling early
time to market
Building a GxP Compliant Cloud means qualification and
validation of the cloud components themselves
Document Set
1) DMS & LMS systems are the compliant
1. Validation Plan (VP)
tooling to ensure reliability and integrity of
data and training requirements 2. System Requirements Specification (SRS)
2) Incident, Problem & Change Management
3. System Configuration Specification (SCS)
System
3) Bare Metal Server – qualified and controlled 4. Traceability Matrix (TM)
hardware configuration through to the
X
physical switches 5. IQ Test Plan (IQP)
4) Datacenter Qualification – SoftLayer facility 6. IQ Test Scripts (IQTS)

qualification to enable readiness of life
7. IQ Summary Report (IQSR)
science audits
5) Core Cloud Technology – openstack / 8. UAT Test Plan (UATTP)
VMWare 9. UAT Test Scripts (UATTS)
6) Disaster Recovery – Resiliency services
10. UAT Summary Report (UATSR)
11. Validation Summary Report (VSR)
12. System Commissioning Memo (SCM)
13. Standard Operating Procedures (SoP)

...
IBM Watson Health Cloud for Life Sciences Compliance (LSC)
Overview
Client users and systems – VPN access
Client, partner and IBM applications

Client
Client
Client Operating
Operating Systems
Systems and
and Applications
Applications Managed
Validated
Validated Systems
Systems and
and Tools
Tools IBM
Managed
Qualified
Qualified and
and Validated
Validated Cloud
Cloud Infrastructure
Infrastructure
Dedicated
GCCI 3CI
Compute Storage
LSC Overall Flow GCCI 2
Compute Storage
Document
GCCI 1
Management Compute Storage
System OpenStack
Learning
OpenStack
Management
System
OpenStack
GMCI
Compute Storage
SCCD
Customer AET
1 One GCCI dedicated to one
OpenStack customer
Shared between customers
Flow
1. Customer logs into SCCD, triggers Service Request via Service Catalog
2. Service Request execution calls AET
3. AET selects customer GCCI
4. openstack API is called by AET
5. AET waits for completion of openstack request
6. AET triggers SCCD workflow continuation
7. SCCD workflow completes
LSC Solution Technical Components
Validated
Validated Systems
Systems and
and Tools
Tools
– Document Management System - opentext Document
Management
System
• Hosted on SoftLayer Learning

Management
System
– Learning Management System – opentext LearnFlex

• Hosted on SoftLayer
Qualified
Qualified and
and Validated
Validated Cloud
Cloud Infrastructure
Infrastructure
GCCI 3
– Private Cloud – ICOS – IBM Cloud Openstack Services

Comp Storag
ute e
GCCI 2
Comp Storag
ute e
GCCI 1
• Based on SL bare metal servers

Comp Storag
OpenStack
ute e
OpenStack
• Adapted in details to meet GxP requirements GMCI

OpenStack
– Incident / Problem / Change Management – SCCD

Comp Storag
ute e
SCCD
AET
• Hosted on ICOS Cloud Instance OpenStack
• Configured with LSC workflows to meet GxP requirements

– Automated Evidence Tooling - AET
• Hosted on ICOS Cloud Instance
• Provides automated IQ documentation for customer VMs and volumes.
ICOS – Standard Architecture CI 3 CI
Compute Storage
CI 2
Compute Storage
CM CI 1
setup Compute Storage
OpenStack
monitor
OpenStack
OpenStack
– One central management (CM)

– Multiple Cloud Instances (CIs) used as private clouds for customers
• every CI is dedicated to a customer
– Based on SL bare metal
– Based on openstack (IceHouse for ICOS release used by LSC)
– Block Storage based on CEPH cluster
– Compute nodes are KVM hypervisors
– Network based on SDN/VE
ICOS – Qualification Overview CI 3 CI
Compute Storage
CI 2
Compute Storage
CM CI 1
setup Compute Storage
OpenStack
monitor
OpenStack
OpenStack
– CM
• Bare Metal Infrastructure Qualification
• CM installation (partly automated) -> CM Infrastructure Qualification
– Each CI
Documents
1. Validation Plan (VP)
2. System Requirements Specification (SRS)
• Bare Metal Infrastructure Qualification

3. System Configuration Specification (SCS)
4. Traceability Matrix (TM)
5. IQ Test Plan (IQP)
• CI deployment (highly automated) -> CM Operational Qualification and

6. IQ Test Scripts (IQTS)
7. IQ Summary Report (IQSR)
8. UAT Test Plan (UATTP)
CI Infrastructure Qualification 9. UAT Test Scripts (UATTS)
10. UAT Summary Report (UATSR)
11. Validation Summary Report (VSR)
– CI Operations
12. System Commissioning Memo (SCM)
13. Standard Operating Procedures (SoP)
• CI operations (log files, user management, etc) -> CI Operational Qualification
ICOS Qualification - Lessons Learned
– Shared components are cumbersome to qualify / validate
• For the CM, we need to have a ‚GxP CM‘, we cannot use the global ICOS CM
• Devops tool chain, like RTC, Jenkins, chef need to be split up
– Non-qualified / non-validated: RTC
– Qualified / validated: Jenkins and chef
• For this we had to clone the central jenkins server from Austin lab into the CM
– Variations in the underlying SL infrastructure cause re-qualification efforts

• SL is changing HW
– E.g. RAID controller
• Tests needed to be adapted since the way to check the RAID array is dependent on the RAID controller manufacturer
– Disk sizes (hard disks, SSD) – same price but higher capacity
• Needs to be address to by formulating System Requirements Specifications (SRS) flexible (e.g. by adding the term ‚or higher‘)
and avoid putting discrete values in the System Configuration Specification (SCS)
– Test Environments: Change history piles up and at a certain point makes re-runs of
qualification and validation test steps impossible
• Can only be addressed by ‚pristine‘ environments.
– throwing away complete environments (SL accounts) including hardware
– Re-order SL hardware and follow the complete qualification / validation process – although not as an ‚official‘
run.
Standard ICOS GCCI Infrastructure Redundancy Features
for High Availability and Data Path Diversity
–Separate networks for management / control and data traffic

• Each network is dual 10 GbE bonded for high availability and high
bandwidth
–Separate & redundant hardware based Network Gateways using IBM

management and Customer access
• Customer gateway for Customer VPN access & Firewall
• Management gateway for IBM Access and Management
–Three OpenStack controllers configured with redundancy to support

High Availability
–Block Storage based on redundant hardware provisioned for multiple

replicas of data (ceph cluster)
LSC Central Management (CM) Architecture
CM
Bare Metal Server Bare
Metal
OpenStack ITM ITMN and Vyatta
Chef DHCP Jenkins
Controller Server OmniBus
Vulnerability TEM Deployment Vyatta Global Load
Scan Server DB Firewall Balancer
CM-Controller 1
MGW
Jump Tivoli Data Deployment TDI

Jenkins Logstash
Server Warehouse DB Server
CM-Controller 2
Notes:
• Each blue rectangle represents a virtual machine
• Grayed out rectangles are available in ICOS but not used by LSC in this release – they
are shut down to reduce validation efforts.
• MGW: vyatta bare metal used as management gateway routing management traffic to and from
CM, especially to the CIs
LSC GxP Customer Cloud Instance (GCCI) Architecture
GCCI
Bare Metal Server Bare Metal
Load OpenStack OpenStack Tivoli Nova Vyatta
Horizon QPID Nova
Balancer Controller DB Directory
SDNVE
SDN SDN SDN Mongo SDNVE
Ceph GCCI-Compute
Controller Connectivity egw DB MGW
GCCI-Controller 1 Nodes (2-n) MGW
GCCI-Compute
GCCI-Compute
...
Nodes (2-n)
Nodes (2-n)
Load OpenStack OpenStack Tivoli
Horizon QPID
SDN SDN SDN Mongo CGW
Ceph Connectivity
Controller egw DB
CGW
GCCI-Controller 2
Ceph
Load OpenStack OpenStack Ceph
Balancer
Horizon
Controller DB
QPID Cluster
Cluster
Tivoli
Ceph
Mongo
Logstash
GCCI-
GCCI-
Directory DB
Storage
Storage
GCCI-Controller 3
Nodes(2)
Nodes (2)
Notes:
• Standard ICOS setup
• Each blue rectangle represents a virtual machine – GCCI Controller 1 and 2 are setup identically, GCCI Controller 3 has a slightly
different set of VMs.
• Compute nodes run nova and sdnve agents
• MGW: vyatta bare metal used as management gateway routing management traffic to and from GCCI
• CGW: vyatta bare metal used as customer gateway routing management traffic to and from customer VMs on the compute nodes.
LSC GxP Management Cloud Instance (GMCI) Architecture
GMCI
Edge App Edge App

DB2 DB2
Server Server Server Server
GMCI SCCD-Compute Node 1 GMCI SCCD-Compute Node 2
Edge App IQ Document PDF Writer Edge App IQ Document PDF Writer
Server Server Srv (iText) Server Server Srv (iText)
GMCI AET-Compute Node 1 GMCI AET-Compute Node 2
Bare Metal Server Bare Metal Vyatta

Horizon QPID
SDN SDN SDN Mongo

Ceph
Controller Connectivity GCCI-Controller 1 egw DB
GCCI-Controller 1 MGW
MGW

Horizon QPID
SDN SDN SDN Mongo CGW

Ceph
Controller Connectivity GCCI-Controller 2 egw DB
GCCI-Controller 2
CGW
Load OpenStack OpenStack Ceph

Ceph
Horizon QPID
Balancer Controller DB
Ceph
Cluster
Ceph
Cluster
Cluster
GCCI-Storage
Cluster
GCCI-Storage
Tivoli
Directory
Ceph
Mongo
GCCI-Controller
DB
3
Logstash
GCCI-Storage
Nodes (2)
GCCI-Controller 3 GCCI-Storage
Nodes
Nodes (2)(2)
Nodes (2)
GMCI:
• SCCD and AET running on 4 compute nodes like ICOS customer workload
• The SCCD system supports multi-tenancy and segregation of tenant’s data
• The SCCD data is segregated based on the customer or customers assigned to the logged in user.
• The AET executes requests from the SCCD catalog and provides required regulatory documentation
Sample Customer GCCI – Network Overview
Customer admin users
accessing SCCD
LON2P CI69 CI50
Internet
(Central Mgmt) (GMCI) (GCCI)
IBM admin users
accessing LSC Vyatta: Mgmt Gateways Vyatta: Mgmt Gateways
Internet
SCCD – compute
mgw01 Service Catalog mgw01 nodes
lon02mgw001ccz069 dal09mgw001ccz050
Public IP:
www.lsc.ibmcloud.com kvm001
dal09kvm001ccz050
mgw02 mgw02
jmp01 lon02mgw002ccz069 see01 – edge server dal09mgw002ccz050
lon2jmp01pcczra lon02see001ccz069
kvm002
dal09kvm002ccz050
see02 – edge server
Vyatta: Customer Gw. lon02see002ccz069 Vyatta: Customer Gw.
chf01 kvm003
lon2chf01pcczra dal09kvm003ccz050
cgw01 cgw01
lon02cgw001ccz069 scc01 – SCCD server dal09cgw001ccz050
lon02scc001ccz069
to customer VMs
cgw02 scc02 – SCCD server cgw02

lon02cgw002ccz069 lon02scc002ccz069 dal09cgw002ccz050
Mgmt Network
172.20.0.0 Internet SCCD checking for authentication Internet end users accessing customer VM‘s
Operating System
VPN Tunnel to Customer LDAP VPN Tunnel to OpenVPN Access
Customer Site
LSC – Cloud Features
Images
 Images must be KVM based – Windows and Linux images only.
 Images need to be provided by customer and imported from customer provided URL
Virtual machines
 Initial size (flavor) may be upgraded by submitting a service request – add CPU, Memory, Disk Size.
 Two vCPUs per physical core
 Physical servers have redundant power and redundant network connections with 10Gbps NICs
 Accessible through VPN. No inbound public internet access. Outbound on request.
Storage volumes
 Block storage. Size can be determined during provisioning
 Not encrypted. Customer is responsible for implementing encryption as required.
 Life time for Volume is different from VM. If VM is destroyed, block storage is still available
 Volume operations (Create/Attach/Detach/Delete) are included in the service catalog
LSC – Configuration Options
LSC provides dedicated configurable cloud environments in a SoftLayer data center - initially in Dallas and London
Restrictions apply e.g. for scalability due limitations in ICOS – accepted for the minimal viable product approach
Dual 12 core 2.6Ghz Xeon v3
Compute nodes 128 GB RAM
Minimum 2 – Maximum 32 12TB RAID10 storage
1x10GbE network link
KVM hypervisor
Dual-server Ceph block Dual 6 core 2.4Ghz Xeon v3

64 GB RAM
storage cluster 2-8 x 4.00TB SATA III
8 - 96TB (ordered, in 8TB blocks) 2 x 200GB SSD
1x10GbE network link
Vyatta network appliances Connects to client VPN and

2 pairs in HA configuration IBM management networks
OpenStack controllers Manages virtual machines and

3 in HA configuration storage in the private network.
Dedicated Rack and Network switches

Graphic shows minimum configuration
Disaster Recovery – Normal Operation
London Data Center Dallas Data Center

Production Production
CM GMCI CM GMCI
Continous
Replication
Or
SCCD DB SCCD DB
Continous copy
of full /
Other data incremental Other data
backups
Active Stand-By
GCCI GCCI GCCI GCCI GCCI

CUST1-A CUST2 CUST3 CUST1-B CUST4
Disaster Recovery – After Fail Over
London Data Center Dallas Data Center

Production Production
CM GMCI CM GMCI
SCCD DB SCCD DB
Other data Other data
ACTIVE
GCCI GCCI GCCI GCCI GCCI

CUST1-A CUST2 CUST3 CUST1-B CUST4
Definitions/Abbreviations/Acronyms
Abbreviations Descriptions
AET Automated Evidence Tool
CM GxP Central Management site -- monitors all GCCIs
GCCI GxP Customer Cloud Instances – single customer tenant private cloud
GMCI GxP Management Cloud Instance – private cloud that hosts SCCD and AET
ICOS IBM Cloud OpenStack Services
IPsec Internet Protocol security for securely encrypting network traffic
LSC IBM Watson Health Cloud for Life Sciences Compliance
OpenStack OpenStack Cloud Management software
Private Cloud Private Cloud is an ICOS Cloud Instance. GCCI and GMCI are both ICOS Private
Cloud instances.
SCCD Smart Cloud Control Desk
SDN-VE Software Defined Networking for Virtual Environments
VM Virtual Machine
VNID Virtual Network ID
VPN Virtual Private Network using Internet Protocol Security
Vyatta Firewall Gateway
Resources
– LSC Knowledge Center

– FDA
– EMA
Backup Charts
Key Architectural Decisions 1 of 2
For helping regulated workload to move from a traditional data center to the cloud, some key architectural decisions had to be
taken in order to
• Remove Qualification / Validation effort from customer (yellow)
• Reduce and Optimize Qualification / Validation effort for IBM as Provider (green)
Area Traditional DC LSC Cloud

Data center • Audit for complete DC • Audit for 2 SoftLayer DCs. Proposal is to have a 3rd party audit.
• Training for DC staff • By using SL‘s Virtual DC only a subset of staff needs to be trained and a
subset of the infrastructure needs to be qualified.
Server / • Qualification of server • By using only dedicated bare metal servers for each customer and avoiding
Storage and storage other SL services, SL‘s IMS needs not to be validated. Black box approach
infrastructure for IMS validation is not acceptable for compliance, white box approach is not
accepted by SL.
• Qualification is done for private cloud hardware, only.
• Cookie cutter approach: Qualify and validate 2 private clouds (DAL and
LON) and subsequently only perform infrastructure qualification for new
private clouds.
Network • Qualification of all • Encryption of all traffic between TopOfRack switches and customer end
network gear and point plus encapsulation of all traffic between customer dedidcated racks
related configuration avoids qualification of all network gear outside the customer dedicated racks.
• Usage of SDN avoids qualification of virtual network gear within the private
cloud.
Hypervisor / • Mostly VMWare for • KVM with openstack allows a disruptive move to the cloud and a forces a
Virtualization long living VMs more cloud like pattern for regulated workload. Avoids legacy in the cloud.
• Using ICOS based on openstack allows a white box validation.
VMs • Qualification for each • Automated qualification for each new VM using AET. Reduces IQ time
new VM significantly.
• QMS process change to allow for automated IQ.
Key Architectural Decisions 2 of 2
Area Traditional DC LSC Cloud
IPCM • Individual Incident • SCCD with a standard set of validated automated changes for the VM life
Problem Change Mgmt. cycle.
• Qualification, Validation • SCCD is validated once, shared between customers.
of IPCM • Customer segregation within SCCD.
DMS / LMS • Validation of Document • Opentext and Learnflex are validated once, shared between customers.
and Learning Mgmt. • Customer have access to documents related to their private cloud.
System
Middleware • Qualification and • Customer responsible for qualification and validation including OS Images.
and validation including OS
Applications images
Audit • Customer is responsible • Customer needs to internally audit the provider for the stack up to and
for the complete stack including the hypervisor. Proposal is to have a 3rd party audit.
(Monolithic approach) • Customer can re-use existing qualification and validation porcess above the
hypervisor
How IBM meets network qualification requirements for GxP compliance
North <-> South Traffic: Encapsulated with-in Rack, Encrypted outside of ToR
SL Network (Public Net or leased line)
VPN Appliance VPN1

Cust
VPN2
Hypervisor Server
SDN-VE Customer DC
Controller &
SL Public network – accessed by only GW/VPN
GW in HA mode
Hypervisor Customer Data Center with VM-A
subnet (1.1.2.1/24) for Hybrid cloud.
SL Private network
 Traffic destined to Customer DC, will be given to GW for de-

capsulating VxLAN tag and point it to VPN1 (and hence get
encrypted before it leaves the rack).
 The Encrypted traffic will go on SL infrastructure (Private or
Public Network), and will follow general SL routing to reach
VM-A VM-B customer DC.
Hypervisor  At customer DC the Encrypted packet will be decrypted at
VPN2.
VM-A VM-B
 Original packet will be delivered at customer DC.
Hypervisor
VM-A VM-B
Hypervisor Data traffic (original packet encapsulated with VxLan)

Data traffic (original packet encrypted without VxLan)
Data traffic (original packet)
LSC is an Integration Project – Based on these Products:
– Compliance Tools – Cloud System
• Document Management System – OpenText • Data Center, bare metal infrastructure, network – SoftLayer
• Learnng Management System – OpenText • Cloud Operating System – ICOS
LearnFlex – Openstack
• Incident, Problem and Change Management: - – Network: SDN/VE
SCCD – Monitoring – Tivoli
– Automation – chef / Jenkins
– Automated Evidence Tooling
• LSC specific development as a bridge between SCCD and ICOS
Dedicated
Customer 1
service portal IBM Cloud OpenStack Services Central Management

on-prem
environments
Dedicated
Customer 2
service portal
on-prem
environments Customer 1 Customer 2 Customer 3 Customer n
Dedicated
Customer 3
service portal
on-prem
environments Authenticate
using customer
LDAP AET
Dedicated
or SAML
Customer n
service portal
IPCMS DMS
on-prem
environments
LSC Management Cloud
Disaster Recovery - Facts
text
– SLA
• RTO = 24 hours for CM/GMCI + another 6 hours per GCCI
• RPO = 30 minutes
– Switch over is a series of semi-automated tasks
• Switching off network connections to primary site
• Starting LSC services on backup site
• Reattach remaining GCCIs to backup site
– Customer is responsible for DR on application level
• Ordering 2 GCCIs in different DCs
• Replicating data from one GCCI to other GCCI
• Restriction: currently based on public VPN
– Network connection from customer site to both GMCIs is established
during onboarding

HealthCloud and ICOS Qualification Overview

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

HealthCloud and ICOS Qualification Overview

Uploaded by

Copyright:

Available Formats

Health Cloud

Hendrik Wagner, Cloud Architect

Life Sciences Product Development Life Cycle

Drug Preclinical Clinical Marketing Marketed

Cloud designs generally FDA: “The regulatory requirements do not

Good Laboratory Practice (GLP)

Good Clinical Practice (GCP)

Good Manufacturing Practice (GMP)

• The major regulatory agencies globally

Key GxP requirements:

Key challenges of maintaining qualified infrastructure include:

Validation is a process for establishing documented evidence, which provides

Predefined requirements Documented evidence

Qualification is a process to demonstrate and document the ability for the

• Develop Infrastructure Qualification Plan (QP)

Cloud providers might not understand

• IBM Watson Health Cloud for Life Science Compliance (LSC)

• Current Status: LA (Limited Availability)

• Previously developed under TechSales organization, LSC is now

Subject Matter Expertise Tools

Processes & Procedures Training

• DMS: Enables electronic workflows, client

Compute Storage Compute Storage Compute Storage

OpenStack OpenStack OpenStack

Softtlayer Data Center

Customer Customer Customer

Well Defined Responsibility

Setup Efforts Recurring Efforts

Infrastructure Setup Efforts Recurring Efforts

4) Datacenter Qualification – SoftLayer facility 6. IQ Test Scripts (IQTS)

11. Validation Summary Report (VSR)

12. System Commissioning Memo (SCM)

13. Standard Operating Procedures (SoP)

Client users and systems – VPN access

Client, partner and IBM applications

Shared between customers

• Hosted on SoftLayer Learning

– Learning Management System – opentext LearnFlex

– Private Cloud – ICOS – IBM Cloud Openstack Services

• Based on SL bare metal servers

• Adapted in details to meet GxP requirements GMCI

– Incident / Problem / Change Management – SCCD

• Hosted on ICOS Cloud Instance OpenStack

• Configured with LSC workflows to meet GxP requirements

– One central management (CM)

1. Validation Plan (VP)

2. System Requirements Specification (SRS)

• Bare Metal Infrastructure Qualification

4. Traceability Matrix (TM)

5. IQ Test Plan (IQP)

• CI deployment (highly automated) -> CM Operational Qualification and

7. IQ Summary Report (IQSR)

8. UAT Test Plan (UATTP)

CI Infrastructure Qualification 9. UAT Test Scripts (UATTS)

10. UAT Summary Report (UATSR)

11. Validation Summary Report (VSR)

13. Standard Operating Procedures (SoP)

• CI operations (log files, user management, etc) -> CI Operational Qualification

– Variations in the underlying SL infrastructure cause re-qualification efforts

–Separate networks for management / control and data traffic

–Separate & redundant hardware based Network Gateways using IBM

–Three OpenStack controllers configured with redundancy to support

–Block Storage based on redundant hardware provisioned for multiple