Db2 Analytics Accelerator Version MDUG May2019

Db2 Analytics Accelerator V7.
1
Technical Foundation
Jim Knisley jknisley@us.ibm.com
May 15, 2019
IBM z Analytics
IBM z Analytics Db2 Analytics Accelerator
Agenda
• Why Db2 Analytics Accelerator

• Version 7.1 Architecture and Deployment Options
• Version 7.1 Functionality
• Version 5 PTF7 Updates
• Customer Use Cases
2
IBM Z Analytics
Keep your data in place – a different approach to enterprise analytics
• Keep data in place for analytics

• Keep data in place, encrypted and secure
• Minimize latency, cost and complexity of data movement
• Transform data on platform
• Improve data quality and governance
• Apply the same resiliency to analytics as your
operational applications
• Combine insight from structured & unstructured data
from z and non-z data sources
• Leverage existing people, processes and infrastructure
3
IBM Db2 Analytics Accelerator for z/OS

A workload optimized, appliance add-on to Db2 for z/OS that enables the integration of analytic
insights into operational processes to drive business critical analytics & exceptional business value.
SPEED
• Dramatically improve query response – up to 2000X faster –
to support time-sensitive decisions
• Right-time. Low latency. Trusted. Accurate.
SAVINGS
• Minimize data proliferation
• Lower the cost of storing and managing historical data
• Free up compute resources
SIMPLICITY
• Simplify infrastructure, reduce ETL and data movement off-platform
• Non-disruptive installation
SECURITY
• Safeguard valuable data under the control and security of Db2 for z/OS
• Protected. Secured. Governed.
4
Hybrid transaction/Analytical processing

The hybrid computing
Transaction platform on IBM Z
Processing
Supports transaction processing and
analytics workloads concurrently,
efficiently and cost-effectively
Delivers industry leading performance for
Analytics mixed workloads
Workload
The unique heterogeneous scale-out
platform in the industry
Superior availability, reliability
and security
Db2 Analytics Accelerator and Db2 for z/OS

A self-managing, hybrid workload-optimized database management system that runs
every query workload in the most efficient way, so that each query is executed in its
optimal environment for the greatest performance and cost efficiency 5
Db2 Analytics Accelerator Version 7.1

Delivering new flexible, integrated deployment options
High-speed analysis of your enterprise data for real-time insight
under the control and security of IBM Z
• Introduces new flexible, integrated deployment options
• Accelerator on IBM Integrated Analytics System
• Fast, simple deployment on pre-configured hardware and software
• Flexible and elastic data storage
• Accelerator on IBM Z
• Unified homogeneity of service, support and operations
• Flexible Capacity
• Based on IBM’s premier analytical engine, Db2 Warehouse software
• Transition easily between deployment options
• One API
• One database engine
6

Deployment Options
In Version 7.1, Db2 acceleration can be implemented
within different operating environments:
• On an on-premises appliance
• On a SW appliance installed on the z14 mainframe
• Db2 Analytics Accelerator Both new options will offer
Version 7.1 • the same functionality
• Two deployment options
• the same API
• the same implementation
This provides:
• Coexistence and combination of deployment
options, fully transparent for Db2 applications
• Flexibility in moving data for query acceleration as
workload demands grow or change
• Consistency and efficiency in managing different
Db2 Analytics Accelerator environments
7

Long and short names
Long name: Db2 Analytics Accelerator for z/OS Version 7.1, deployment on IBM
Integrated Analytics System (IAS)
Short name: Accelerator on IAS
Long name: Db2 Analytics Accelerator for z/OS Version 7.1, deployment on IBM Z
Short name: Accelerator on IBM Z
8
Agenda
• Value Proposition
9
10
Db2 Analytics Accelerator
The core of V7.1 Accelerator Deployments

Docker is a widely adopted, open-source project that automates
Docker container the deployment of applications inside software containers.
Accelerator Database Additional future
server engine functionality It allows to bundle and preinstall components in a Docker image
and then to launch the container from the image.
Systems Workload
Authentication
Manager Monitoring The accelerator Docker container includes
Infrastructure • the accelerator server that establishes the connection
Management
Linux between the Db2 for z/OS subsystem and manages all
accelerator tasks
Physical server (Power) CPU Memory • a database engine
• other components required for high availability,
Storage (SAN) (Clustered) Filesystem monitoring and security
• a Linux operating system
Outside of the container physical compute resources are needed
• a server that includes multi-core CPUs
• large memory
• shared filesystem to persist the data
On top of the physical hardware there is a Docker supported
Linux operating system that is used to launch the Docker
container and manage the HW resources.
10
Powered by Db2 with BLU Acceleration
In-memory column processing Multi-core and SIMD Patented compression Skips unnecessary
with dynamic movement of data parallelism technique preserves order -- processing of irrelevant data
from storage (Single instruction data can be used without
Multiple Data) decompressing
• Huge potential for faster ingest for incremental updates, and thereby less HTAP query delay!
• IBM’s premier analytics engine across many products
• Latest analytics technology innovations
• SQL compatibility across all IBM products
• High degree of concurrent users and queries
The new engine is replaced internally – external interfaces will stay the same.
The same Db2 subsystem can be connected to the existing and new generation.
11
Db2 Analytics Accelerator V7.1, deployment on IBM Integrated

Analytics System (IIAS)
• New generation hardware appliance

• A full solution that provides all components out of the box – including optimized hardware and software
• All components provided by IBM in a balanced, performance-optimized configuration
• HW, which includes the rack, the physical servers and the storage
• SW stack including the Linux operating system, the Docker software as well as the Docker container
and the infrastructure management
• IBM Power hardware for the appliance, balanced and optimized for price/performance
13
IBM Db2 Analytics Accelerator on IIAS

Product components
IBM Z IBM Integrated Analytics System
CLIENT
OSA-Express
• Data Studio with Db2 Analytics
10 GbE
Accelerator Studio Plug-in
Patch Panel
• Data Server Manager 2.1.5 or higher
OSA-Express
10 GbE
Dedicated highly available
network connection
IBM Db2
Users/ Db2 for z/OS enabled for Analytics
Applications IBM Db2 Analytics Accelerator Accelerator
incl. Stored procedures
14
IBM Integrated Analytics System Configurations
M4002-003 M4002-006 M4002-010 M4002-020 M4002-040

1/3 Rack 2/3 Rack Full Rack Double Rack Quad Rack
Servers 3 5 7 13 25
Cores 72 120 168 312 600
Memory 1.5 TB 2.5 TB 3.5 TB 6.5 TB 12.5 TB
Flash Storage 27 TB 54 TB 81 TB 162 TB 324 TB
Capacity
IBM Power 8
S822L 24 core
server 3.02GHz 15
Hardware Architecture Overview – Full Rack M4001-010

7 Compute Nodes in 1 rack containing
• IBM Power 8 S822L 24 core server 3.02GHz
• 512 GB of RAM (each node)
• 2x 600GB SAS HDD
User Data Capacity:
• Red Hat® Linux OS 192 TB*
(Assumes 4x compression)
Up to 3 Flash Arrays in 1 rack containing
• IBM FlashSystem 900
• Dual Flash controllers Power Requirements:
• Micro Latency Flash modules 9.4 kW
• 2-Dimensional RAID5 and hot swappable
spares for high availability
2x Mellanox 10G Ethernet switches Cooling Requirements:

• 48x10G ports 32,000 BTU/hr
• 12x40/50G ports
• Dual switches form resilient network
IBM SAN64B 32G Fibre Channel SAN Scales from:
• 16Gb FC Switch 1/3rd Rack to 8 Racks
• 48x 32Gb/s SFP+ ports
(initial GA is 1/3rd to 1 Rack)
16
IBM Integrated Analytics System – Designed for speed

Powered by RedHat® Linux on Power
Optimized for analytics with 4X threads per core, 4X memory
bandwidth and 6X more cache at lower latency compared to x86
ALL Flash Storage

Hardware accelerated architecture enabling faster insights with extreme performance,
99.999% reliability and operational efficiency
Designed for Massively Parallel Performance

Memory Optimized
In-memory BLU columnar processing with dynamic movement of
data from storage
Data Skipping
Skips unnecessary processing of irrelevant data
Actionable Compression
Patented compression technique that preserves order allowing data to be used
without decompressing first
17
Db2 Analytics Accelerator Version 7.1, deployment on IBM Z

IBM Secure Service Container
Docker container
Components
Existing
Accelerator Db2 Warehouse Additional future
server engine functionality
SE / HCM Workload
Systems Manager Authentication
Monitoring
Accelerator Solution
Delivered as part of
Docker supported OS + management
PR/SM LPAR CPU Memory
Customer’s
Storage Storage (SAN) Filesystem
Management
• An appliance running on IBM Z

• Packages the SW stack into an IBM Secure Service Container to deliver a fully self-managed appliance
running in a SSC LPAR that can be deployed in minutes
• Integrates seamlessly into the customer’s Z environment and leverages known LPAR-, memory and CPU
management procedures, including call home support for enterprise hardware components.
• Uses customer-provided storage to hold the accelerator-side data
18
Deployment of an Accelerator Image into an IBM Z Secure

Service Container (SSC) LPAR
SSC Partition & Deployment Service
• New SSC partition mode for SW appliances
• SSC partition is not for general purpose usage, only appliances can
be deployed
• Speeds up deployment with an integrated installer
• Ensures integrity of downloaded appliance
SSC Appliance Runtime
• Operating system incl. Docker runtime
• Network and storage control incl. encryption
• System monitoring and FFDC
SSC Appliance Management Services
• Admin/User controls
SSC LPAR
Accelerator on Z – View messages, events
– Network management
Image – User management
– Disk management
– View Appliance status
– Create/obtain dumps/logs
IBM Z Firmware SSC Installer and Loader – Apply service, updates
• All management interactions with an appliance are via Web UI /
REST APIs
• Software appliances based on SSC technology include zAware,
zVSE, Blockchain, … 19
Hardware Considerations
• 1 Accelerator = 1 LPAR – but multiple Accelerator LPARs on a single system possible
• Each LPAR requires IFLs, RAM, and storage
IFLs Dedicated z14 Drawer Storage

Use existing IFLs and memory • Order one drawer, comprised of 35 • Customer provided storage
For small production test/dev or IFLs and up to 2.56 TB memory, with (actual size depends on
getting-started use cases your z14 order workload)
Minimum suggested configuration: • You can convert your z14 models • FCP or FICON attached
M01, M02, M03 servers by adding a • Flash storage is not
• 4 IFLs, 256 GB memory for dedicated drawer containing IFLs
test/dev required, but e.g. IBM
and memory at a very attractive price FlashSystem 900 is an
• 8 IFLs, 512 GB memory for • Also available on M04, M05 as a new effective option
production build inclusive of the drawer, as you
cannot add another drawer to M04
or M05 machines
20
Hardware Considerations (contd.)

• Accelerator software supported on IBM Z starting with z13
• IBM z14 Type 3906 Models M01, M02, M03, M04 and M05
• IBM z14 Type 3907 Model ZR1
• IBM z13 Type 2964 Models N30, N63, N96, NC9 and NE1
• IBM z13s Type 2965 Models N10 and N20
• IBM LinuxONE Emperor II Type 3906 Models LM1, LM2, LM3, LM4 and LM5
• IBM LinuxONE Emperor Type 2964 Models L30, L63, L96, LC9 and LE1
• IBM LinuxONE Rockhopper II Type 3907 Model LR1
• IBM LinuxONE Rockhopper Type 2965 Models L10 and L20
• Check with IBM for required firmware levels
• Accelerator runs in SSC LPAR
• 1 Accelerator = 1 LPAR – but multiple Accelerator LPARs on a single system possible
• Each LPAR requires IFLs, RAM, and storage
• IFLs can be shared
• Not under z/VM or KVM
21
IBM Db2 Analytics Accelerator on IBM Z

Product components
Db2 code
IBM Z
including Stored Procedures
Appliance UI
Private
network
• Data Studio with Db2

Analytics Accelerator Accelerator Appliance
Studio Plug-in • Can be deployed on the
• Data Server Manager same CEC as Db2 or a
2.1.5 or higher different one 22
Network Connection to Db2 for z/OS
A defined TCP/IP network

interface from one Accelerator to
Db2 subsystem(s)
– HiperSocket (CEC-internal)
OSA OSA
or
HiperSocket HiperSocket – OSA (internal or external)
M:N connectivity
– Multiple Db2 subsystems can
share an Accelerator
– Multiple Accelerators can
OSA 10 Gbps recommended
connect to a Db2 subsystem
Shared OSA is possible
23
Deployment Considerations – Accelerator Instances

• Db2 Analytics Accelerator on IBM Z instances can be deployed quickly (download & go)
• Instances are currently limited to CPU resources in one physical drawer
• Higher instance granularity is possible than on IIAS based systems
§ Multiple independent applications / Db2 subsystems may share a bigger IIAS system while
Accelerator on Z characteristics may lead to more, dedicated and smaller instances
§ Beneficial for (many) smaller test/dev instances which could be temporarily (re-) provisioned
Db2 Analytics Accelerator Db2 Analytics Accelerator on Z
On IIAS (e.g., full rack)

Db2 Analytics Accelerator on Z
24
Summary: One API – One implementation – Two deployment options

Hardware Appliance z based Appliance
§ Uniform experience, simultaneous use, and easy transition
between different implementations
§ Common accelerator engine across all the platforms:
Db2 Warehouse
POWER hardware and storage integrated in a self-contained Software appliance deployed on customer’s z hardware and
workload-optimized system for analytics storage infrastructure
Out-of-the-box experience Download & go experience
POWER-grade QoS Z-grade QoS
Scalability by scale-out Scalability by using Z: expansion of IFLs and memory
HA support within appliance CA/DR support based on GDPS (active/passive and active/active)
Analytics for largest data volumes and highest performance Flexible, smaller, elastic deployment option
Integrated storage and management integration into existing z environment: hardware and storage
management, CA/DR infrastructure, support processes,
organizational structures,… – no new infrastructure needed 25
Connectivity options
Multiple Db2 systems can connect
to a single Accelerator
A single Db2 system can connect

to multiple Accelerators
Multiple Db2 systems can connect

to multiple Accelerators
Represents all
deployment
options
Full flexibility for § residing in the same LPAR § being independent (non-data sharing)
Db2 systems and § residing in different LPARs § belonging to the same data sharing group
deployment options: § residing in different CECs § belonging to different data sharing groups 26
Deep Db2 for z/OS integration

Applications DBA Tools, z/OS Console . . .
Application Interfaces Operational Interfaces
(standard SQL dialects) (e.g. Db2 Commands)
Db2 for z/OS

IBM Db2
Data Buffer ... Log Analytics
IRLM
Manager Manager Manager Accelerator
Superior
availability,
reliability, security
Workload
management
Superior
z/OS on IBM Z performance on
analytic queries
27
Agenda
28
IBM z Analytics
Functionality overview (V7.1.5)

• Query routing and acceleration
• Data synchronization
– Db2 table and partition load
– Incremental update
– Db2 Analytics Accelerator Loader
• „True HTAP“ – Data coherency between Db2 and Accelerator
• Accelerator-only table usage

– In-database transformation
– Multi-step reporting
– Storing external or historical data
29
Query execution process flow

Heartbeat
(availability and performance indicators)
Application
Interface Optimizer
Heartbeat
Queries executed
without Accelerator
Accelerator DRDA Requestor

Query execution run-time for queries
that cannot be or should not be
routed to Accelerator
Application
Queries executed
with Accelerator
30
Routing criteria
§ Dynamic and static queries can be § Prerequisites for query routing:

accelerated • Accelerator is started
§ Db2 Optimizer decides if query should be • All used tables are available on Accelerator
sent to Accelerator • Query routing option QUERY ACCELERATION
• Dynamic: At execution time is specified
• Static: At BIND time § Via special register, BIND option or ZPARM
§ Whole query, not parts of query are § Values: ENABLE,
accelerated ENABLE_WITH_FAILBACK, ELIGIBLE, ALL,
§ Only read queries are considered for NONE
acceleration
§ Queries within INSERT statements can be
accelerated 31
SQL functionality support and restrictions

•Improved Db2 for z/OS SQL support on Accelerator V7 (compared to Accelerator V5)
• All data types supported except LOBs or XML
• Improved Db2 for z/OS function support on the Accelerator
• Still some not supported, e.g ACOS, ASIN, CLOB, ..
• Correlated subquery support
• Recursive SQL support
• Special register support
• Passthrough support for a set of OLAP built-in functions available on the accelerator but not on Db2 for z/OS
(requires V7.1.5)
•Restrictions:
• No user defined functions (except inline SQL scalar UDF, compiled SQL scalar UDF)
• No multiple encoding schemes in the same statement
Knowledge Center: Conditions for query routing to an accelerator

https://www.ibm.com/support/knowledgecenter/en/SS4LQ8_7.1.0/com.ibm.datatools.aqt.doc/gui/concepts/c_idaa_qu
ery_offloading_criteria.html
32
Data load and update options with Db2 Analytics Accelerator

Synchronization options Use cases, characteristics and requirements Technical aspects
Full table load/refresh § Source table data is entirely replaced

The entire content of a § Smaller, un-partitioned tables § Scope: Table or Partition
database table is § ACCEL_LOAD_TABLES stored procedure
loaded/refreshed § Reporting based on consistent snapshot
§ Data Studio provides options to
§ Load/Refresh a table/partitions
Table partition § Indicate changed partitions
load/refresh § More efficient than full table refresh for larger tables
§ Queries can be routed while load is in
For a partitioned database § Reporting based on consistent snapshot progress
table, selected partitions § Optionally: automatically load changed partitions only
can be loaded/refreshed
§ Scope: Row
Incremental Update § Scattered updates after “bulk” load § Based on Change Data Capture (CDC) of IBM
§ Reporting on continuously updated data (e.g., an ODS), InfoSphere Data Replication
Log-based capturing of
changes and propagation to considering most recent changes § Management integrated into stored
Accelerator with low latency § More efficient for smaller updates than full table procedures and Data Studio to:
(typically few minutes) refresh • Enable/Disable tables for replication
• Start/Stop replication 35
IBM z Analytics
Accelerator data load

§ Scope: Table or Partition
§ Data is loaded or refreshed using ACCEL_LOAD_TABLES stored procedure

• Non-partitioned tables and table partitions are loaded in parallel
• AQT_MAX_UNLOAD_IN_PARALLEL determines maximum number of load streams
• Accelerator smartly manages the number of streams to ensure high performance and prevent overloading
§ Queries can be routed to accelerator when tables or partitions are being refreshed
§ Changes to DB2 data can prevented during load process using LOCKMODE option
• LOCKMODEs supported: TABLESET, TABLE, PARTITION, NONE
• Ensures consistent snapshot
§ Supports change detection: DB2 automatically determines if table/partition was changed based on DB2 real
time statistics
• Otherwise skips the table/partition in the load request 36
Accelerator data load
Table B Table A
Db2 Analytics Accelerator Administrative

Part 1 Unload USS Pipe
Table C
Stored Procedures
Part 2 Unload USS Pipe

Table D
Db2 Analytics Part 1

Accelerator . . .
Studio . . .
Part 2
. . .
Data
Part 3
Part m Unload USS Pipe Slices
38
IBM z Analytics
Incremental Update
§ Keeps data in DB2 and the Accelerator in synch in near real-time
§ Scope: Row
§ Based on the Change Data Capture (CDC) component of IBM InfoSphere Data Replication
§ INSERT/UPDATE/DELETE statements captured from DB2 log data and replicated
to the Accelerator
§ Default apply interval around 31 secondes
§ UPDATES are decomposed into DELETEs and INSERTs
§ Tables enabled for incremental update require either an enforced uniqueness (primary key,
unique index) or a defined informational constraint (via ACCEL_ADD_TABLES stored procedure)
§ Required for DELETEs
§ Continuous replication
§ Base table not locked while table initially loaded to the Accelerator
§ Replication not stopped if replication subscription is changed (tables added, removed, loaded, reloaded)
39
Incremental update architecture

Applications executing Accelerator Users
I/U/D Statements on enabling tables
replicated tables for replication
Stored Accelerator
Table Procedures Server Table
T1 T1
Table Table
T2 T2
Manages
Table Subscription CDC Table
T3 Access T3
Server
T T T
1 2 3
Db2 Log
Data
CDC CDC
Capture Replication
Agent Committed Data
Engine
40
Db2 Analytics Accelerator Loader for z/OS*
• Significant cost & time reduction by

eliminating manual ETL processes of non-Db2
data
• Insight into more data types such as IMS,
VSAM, sequential files, Oracle, Adabas, etc.
• Ensure High Availability for critical analytics
applications
• Protect critical analytics data with backup &
restore strategy
Extend analytic capabilities by bringing
• Support cascading changes made to replicated non-Db2 data to IBM Db2 Analytics
tables to reduce data latency Accelerator and IBM Z
• Enhance use of Accelerator-only tables with
one-step load and load resume
*Separately licensed option
Db2 Analytics Accelerator Loader

Extends Capabilities of the Accelerator beyond Db2 data
Remote
SMF Data VSAM & Flat File
Db2 for z/OS
Db2 LUW Oracle ADABAS
Other Relational
z/OS Syslog IMS
non z
Now supporting IDMS and comma-separated values


Automates loading of non-Db2 data
Process to identify and

map source data
Entire process is automated in one
job
• Create Db2 table
• Table added to Accelerator
• Specified source data extracted
• In-memory conversion to Db2
format
• Data loaded to Accelerator
• Table is ready for acceleration
Db2 Analytics Accelerator Loader Loads to multiple

High Availability Load Accelerators are
performed serially
#1
S-
A BLE
D_T
OA
L_L
T CE
AC
A
B
L
E ACCEL_LOAD_TABLES - #2
AC
CE
L_
LO
AD
_T
AB
LE
S-
#3
PI59666: HA Load from Db2 for z/OS or external source

PI67510: Specify accelerator group name
Db2 Analytics Accelerator Loader Loads to multiple

High Availability Load Accelerators are
performed in
parallel
T
A
B
L
E High Availability Load
A one single load operation
1. Reduced Elapsed Time

2. Reduced CPU
3. Simplified Operations
PI59666: HA Load from Db2 for z/OS or external source

PI67510: Specify accelerator group name

Accelerator Backup and Recovery
Expanded Data Load Capability Plus Data Protection
Makes sense for
– Data loaded to AOT or accelerator only
– Data that has been changed or altered in AOT
– Data loaded with LOAD RESUME over time
Allows
– Protection of investment spent in loading data
– Protection for data changed in an AOT
– Ability to potentially reduce backup of data at
its source
Features
• Included with Accelerator Loader product
• Integrated backup, and fast recovery when needed
• Familiar DBA Functionality
• Fits into disaster recovery scenarios
PI70981: Backup & Recovery of Accelerator-only Tables
• Fast Restore to copy point
47
What is True HTAP and why ? Available with V7.1.4

• Single System, Single Interface
Tra
ns
Pro action
ces a
• Data coherency between Db2 and Accelerator sing l
• Don’t worry about latency of committed data

Queries run first when data available
• No longer need to explain to users why committed data not seen.

Data used by SELECT contains corresponding commits
• e.g. Use case ”Fraud Detection” al

lytic
Detection query works on most recent data Ana ssing
ce
Pro
• e.g. Use case “Multi form consistency”

Enter data in a field on a panel. Application commits data.
=> Just entered data shown on following panels
47
48
How does True-HTAP work?
Write
requests
Asynchronous replication
OLTP
reads
Most recent
OLAP committed
reads data no
required? yes
yes
Most recent
committed no Initiate
data apply
Wait for committed data available?
from time of SQL request

Wait for
given time
period
48
49
How does HTAP work?

• Introducing new zParm QUERY_ACCEL_WAITFORDATA + Special register
– CURRENT QUERY ACCELERATION WAITFORDATA = n Write
• n = 0 - 3600 (seconds) requests

Asynchronous
• Default: 0 = No wait
replication
OLTP
• Important: Can be set differently for each query

reads
Most recent
OLAP committed
reads data no
– WAITFORDATA = 0 required? yes
yes
• Immediately execute in accelerator (Current behavior, no delay)
Most recent
committed no Initiate
data apply
available?
– WAITFORDATA > 0 Wait for

given time
period
• Wait for committed changes to be applied via asynchronous replication

• If wait time is exceeded check Delay Expiration rule (Option 13 on Configuration Console):
o If “FAIL” (default), check CURRENT QUERY ACCELERATION special register
Ø If “WITH FAILBACK” is specified, execute query in DB2
Ø Else FAIL query with SQLCODE -904
o If “CONTINUE”, immediately execute in accelerator and return SQLCODE +904
49
How does HTAP behave?

Execute on
WAITFORDATA Accelerator with
N
>0? "old" data
SQLCODE 0
Execute on
• The flow diagram shows the effect of the Timeout before N
Accelerator with
WAITFORDATA special register data arrived? "new" data
SQLCODE 0
Y
• It shows how the Accelerator processes a query,
dependent on settings and timings Execute on
Accelerator with
OPTION 13 N "old" data
= Fail ?
SQLCODE +904
CURRENT QUERY Fail Query

ACCELERATION = N
"WITH FAILBACK" SQLCODE -904
Execute on
DB2
50
Infos & Best Practices for True-HTAP ( WAITFORDATA > 0)

• WAITFORDATA > 0 ensures that a query sees all committed data.
This is done by executing additional steps in the CDC stream and applying data as
needed.
• Many complex report queries need data from the previous hours but not the last
minute.
These do not need WAITFORDATA to be set
• For good response times and replication latency, we recommend limiting queries with
WAITFORDATA > 0 to a few per minute
• Spikes and high update and delete rates in replicated tables will impact True HTAP
query response times
• The query rate can be increased if response times of many minutes are acceptable for
ALL queries
• At the current time HTAP can be used for dynamic SQL
51
Introducing Accelerator-only table type in DB2 for z/OS

Creation (DDL) and access remains through DB2 for z/OS in all cases
Non-accelerator DB2 table

• Data in DB2 only Table 1
Accelerator-shadow table
Table 2 Table 2
• Data in DB2 and the Accelerator
Accelerator-archived table /
Not partition Table 3 Table 3
available • Empty read-only partition in DB2
for V7 yet • Partition data is in Accelerator only
Table 4 Table 4
Accelerator-only table (AOT)
• “Proxy table” in DB2
• Data is in Accelerator only
Accelerator-only tables
Supporting in-database transformation and multi-step processing
Introduction of Accelerator-only tables (AOT)

to store intermediate or final results of data
transformation or reporting processes
§ Can be used to store a set of data in Db2 Analytics Accelerator only, not on Db2 for z/OS, without using
the High Performance Storage Saver functionality
§ Accelerate in-database data transformations and data movement processes
§ Reduced need of data movement processes to other platforms for data transformation purposes
§ Enables multi-step reporting on the Accelerator
§ Saves disk space and CPU cost on IBM Z currently used for transformations and reporting steps
§ Allow data preparation steps for data mining and other advanced analytics to execute on the
Accelerator
53
Accelerator-only tables – Technical basics

§ AOTs are created and dropped using Db2 DDL statements (CREATE; DROP)
• Accelerator must be started
• QUERY ACCELERATION behavior may have any value during CREATE/DROP
• Syntax:
§ CREATE TABLE MYTABLE (...) CCSID ccsid IN ACCELERATOR <ACCEL1>;
§ DROP TABLE MYTABLE;
§ Recommended to create a database in Db2 to be used for the AOTs

• CREATE TABLE MYTABLE (...) CCSID ccsid IN ACCELERATOR <ACCEL1> IN DATABASE MYDB;
• Usual authorization necessary to create objects in database
§ SELECT and INSERT/UPDATE/DELETE operations using AOTs can only run on the Accelerator
• QUERY ACCELERATION behavior must be set to ENABLE/ELIGIBLE/ALL
• Accelerator-shadow tables and other AOTs can be used in the same statement
§ Dynamic and static SQL can be used with AOTs 54

Multi-step reporting applications with Db2 for z/OS

With Accelerator-only tables: Temporary objects and processing on the Accelerator
Reporting Application
Multi-Step Report Credit Card

Credit Card
Transaction History Transaction History
1
Customer Customer
Summary Mart Summary Mart
2
1
Temporary
n results
2
Temporary
results
n
Temporary
results
Reports and Dashboards

Data for transactional and analytical processing 55
Using Accelerator-only tables and ELT logic in the

Accelerator (In-database transformation)
Customer Customer
Transactions Transactions
Transaction Processing Customer Customer

Systems (OLTP) Data Data
ELT logic
Customer Transaction
Summary and History
AOTs
Customer Summary
Analytics Mart AOTs
Advantages:
• Simpler to manage
• Better performance and
reduced latency Data for transactional and analytical processing
56
IBM z Analytics
IBM Db2 Analytics Accelerator Version 7.1

Continuous Delivery – Fixes and new Features every other Month (on avg.)
Enhancements in 7.1.2 and 7.1.3 levels
• Incremental Update
• New datatypes: BINARY, DECFLOAT, ROWID
• Recursive SQL support
Accelerator on IBM Z • Support for Db2 Analytics Accelerator Loader
• Accelerator monitor in Data Studio
• Configuration Console access via ssh
• Support for additional IIAS hardware sizes and
configurations
– M4002 series including M4002-003 GoD
• Additionally supported IBM Z hardware
– z14 ZR1
– z13 and Linux One (check firmware levels with IBM)
Accelerator on IIAS (Appliance)
57
IBM z Analytics

Continuous delivery – fixes and new features every other month (on avg.)
Enhancements in 7.1.4
• True HTAP support
• Pass-through support for a set of OLAP built-in functions
Accelerator on IBM Z
• Support for Local Date Format in accelerated queries
• Support for multi-row insert for accelerator-only tables
• True HTAP: SMF counters for monitoring wait/expiration times of queries
• Accelerator on Z:
• GDPS support for fail-over scenarios
• Script based image deployments
• Coexistence V5/V7: Option to specify default accelerator for query routing
Accelerator on IIAS (Appliance) to support migration scenarios (Db2 enhancement only)
58
IBM z Analytics

• Call-Home for IIAS via HMC (HTTPS proxy on HMC)
• Encryption of Data in Motion for all network traffic, now also Incremental
Update traffic
Accelerator on IBM Z Enhancements in 7.1.7

•High Performance Storage Saver (HPSS) as Technical Preview
• Supported use cases within the Technical Preview:
• Archiving partitions of range-partitioned tables
• Restoring partitons
• High availability: Archiving to multiple accelerators
• Migration/Coexistence: Archiving of partitions to Accelerator
V7 that are already archived on Accelerator V5

59
IBM z Analytics

•Compression improvements via full table reload
• If you use Accelerator V7, but experience a low compression of tables on
the Accelerator then with V7.1.7 a full table reload will improve
compression significantly.
Accelerator on IBM Z •Support for partial reloads of replicated tables
• If a table has been enabled for replication then you can now execute a
reload of partitions (e.g. with change detection) on this table using
ACCEL_LOAD_TABLES stored procedure while this table is still enabled
for replication. This could be useful if fr example a batch job changed a
large number of partitions of that table. Loading the data of the changed
partitions in many cases is more effective in terms of speed and
resource consumption than replicating the data.

60
IBM z Analytics

•Passthru support for additional built-in functions. See this blog entry:
Newly supported builtin functions in Db2 12 for z/OS with Db2 Analytics Accelerator V7
•Accelerator on Z: Support for small deployments for small production, test/dev or
getting-started use cases. The minimum suggested configuration is: 2 IFLs, 64 GB
Accelerator on IBM Z
memory and 16 GB for each Change Data Capture (CDC) subscription (in case
Incremental Update is used).

61
IBM z Analytics
Current Hardware Status

Product name General availability End of Support Accelerator version
PureData System for Analytics November 2011 June 30, 2019 Up to V5
N1001
PureData System for Analytics February 2013 June 30, 2019 Up to V5
N2001
PureData System for Analytics May 2015 June 30, 2020 Up to V5
N2002
PureData System for Analytics October 2014 Not announced yet Up to V5
N3001
Integrated Analytics System September 2017 Not announced yet V7 only

M4001
Integrated Analytics System March 2018 Not announced yet V7 only
M4002
62
IBM z Analytics
What’s next?
A brief outlook on new capabilities

on the roadmap for 2019
Note the disclaimers on unannounced product features!
63
Roadmap
Very Large Accelerator Deployments

Background
- IIAS accelerator deployments are available in different sizes, ranging from
M4002-002GoD (1/3-rack “growth-on-demand” with 3 capped server
nodes) to M4002-020 (double-rack with 13/14 server nodes)
- Z accelerator deployments are supported from very small (2 IFLs, 64 GB)
to large (~40 IFLs, ~4 TB)
- This fully covers many accelerator deployments, but presents a limitation for
growth, since 32 IFLs / 2.5 TB provide performance/capacity “similar” to
M4002-003
è The new “Multi-Node Deployment Option on IBM Z” will close that gap and
is designed to scale up to performance/capacity similar to M4002-020
Roadmap
New Deployment Options on IBM Z

• Single-node (1 LPAR) Multi-node (multiple LPARs)
• One combined data / head Multiple data nodes + one head node
node - Scaling by changing size of nodes
- Data nodes: 6 – 40 IFLs, 512 – 4000
- 2 – ~40 IFLs, 64 GB – ~4 TB GB
• Up to 20 TB uncompressed - Head node: 4 – 32 IFLs, 256 – 1000
GB
Db2 - Limited only by Z system capacity (z14 max 170 IFLs total)
– Less if special performance Up to 100 TB uncompressed Db2
requirements, e.g., IU latency,
Less if special performance
query elapsed time,... requirements, e.g., IU latency, query
• Ideal for “white space” elapsed time,...
(unused capacity) Ideal for deployments on “dedicated”
deployments in existing CPCs systems, e.g., LinuxONE
Roadmap
Multi-Node Deployment – Architecture

Accelerator on Z Z CPC
HiperSocket
Head Data 1 Data N
…
Db2 z/OS networ IDAA server Db2 WH Db2 WH
k Db2 WH • Data • Data
• Catalog partitions partitions
• No data
partitions
LPAR LPAR LPAR
…
Storage
Roadmap
Multi-Node Deployment – Advantages

• Scalability of the Accelerator on Z for the most demanding
workloads
• Multi-node accelerator can grow from the entry level (~30 IFLs) to
the largest size using all available IFLs on a system without ever
reloading the data
• Extremely flexible adjustment of resources (IFLs, memory, storage)
to optimize for the actual workload requirements
– Even dynamic adjustments (add/remove IFLs, add/remove memory, add
storage) are supported and require only short or even no downtime
• Maintains all advantages of the deep integration into the Z platform
Roadmap
Multi-Node Deployment – Q&A

• Why are multiple LPARs used instead of just increasing LPAR size?
• Performance measurements have shown that the
performance/throughput for a multi-LPAR configuration is better per
IFL than scaling a single LPAR far beyond 40 IFLs
• Current limitation of a single SSC LPAR to 128 IFLs and 10 TB
memory
• Does the Z system running the accelerator need to be dedicated?
• No. Other workloads can run on the same system. However, all of
the accelerator LPARs need to run on the same system – which may
mean a significant portion of the overall capacity.
• A single system can also run multiple accelerators – using both
deployment options (single-/multi-node) in the same environment
Roadmap
Integration with GDPS Metro Global – GM

• Currently, the Accelerator on Z is supported by GDPS Metro (see
above)
• Many clients have requested to extend this support to global
coverage in order to be able to fully integrate the accelerator into
their existing HA/DR implementation
è GDPS Metro Global needs to be extended to support the
accelerator
This integration will offer
• 3-site support
• 4-site support
• DR test support based on flashcopy
Roadmap
GDPS Metro Global – 3-site Support

Campus – CEC A Campus – CEC B DR Site – CEC C
Data Sharing Group
DB2
CF
GDPS KSys
XRC instance z/OS LPAR B
GDPS RSys
GDPS K1 GDPS K2 XRC instance z/OS LPAR C
Metro instance z/OS LPAR A Metro instance z/OS LPAR B
IDAA-on-Z LPAR C
IDAA-on-Z LPAR A IDAA-on-Z LPAR B
GDPS Agent
GDPS Agent GDPS Agent
nt
iffere
B u ses d PAR A LPAR C uses same
LPAR data as L data as LPAR B
/
Disks
P P P GDPS-PPRC S S S GDPS-XRC DR DR DR
Roadmap
GDPS Metro Global – 4-site Support

Campus – CEC A Campus – CEC B DR Site – CEC C DR Site – CEC D
Data Sharing Group
DB2 DB2
CF
GDPS KSys GDPS KSys GDPS RSys GDPS RSys

XRC instance z/OS LPAR A XRC instance z/OS LPAR B XRC instance z/OS LPAR C XRC instance z/OS LPAR D
GDPS K1 GDPS K2 GDPS K1 GDPS K1
Metro instance z/OS LPAR A Metro instance z/OS LPAR B Metro instance z/OS LPAR C Metro instance z/OS LPAR D
IDAA-on-Z LPAR A IDAA-on-Z LPAR B IDAA-on-Z LPAR C IDAA-on-Z LPAR D

GDPS Agent GDPS Agent GDPS Agent GDPS Agent
nt
iffere
R B uses d PAR A
LPA as L
/data
Disks
GDPS-PPRC XRC PPRC DS DS DS

P P P S S S DR DR DR
Roadmap
GDPS Metro Global – Disaster Recovery Test

Campus – CEC A Campus – CEC B DR Site – CEC C
Data Sharing Group
DB2 DB2 DB2
CF
GDPS KSys GDPS RSys

XRC instance z/OS LPAR B XRC instance z/OS LPAR C
GDPS K1 GDPS K2
Metro instance z/OS LPAR A Metro instance z/OS LPAR B
IDAA-on-Z LPAR A IDAA-on-Z LPAR B IDAA-on-Z LPAR C

GDPS Agent GDPS Agent GDPS Agent
Flash
P P P GDPS-PPRC S S S XRC
DR DR DR Copy DS DS DS
Agenda
73
IBM z Analytics
What’s new in Version 5 PTF 6 & PTF 7

• Hybrid Transactional Analytical Processing (HTAP)
– guaranteed that the result set of queries takes all transactions into
account that have been committed at the point-in-time the query is
run
• Add-column support for replicated tables
– Replicated tables can now be synchronized using the IBM DB2
Analytics Accelerator Studio or the stored procedure
SYSPROC.ACCEL_SYNCHRONIZE_SCHEMA
• Secure FTP support for Call-home
• Federation (now also supported for DB2 12 for z/OS)
74
Agenda
75
devWorks Community for IBM Db2 Analytics Accelerator

Join
ibm.biz/db2analyticsaccelerator
76
Learn more about the Db2 Analytics Accelerator

Visit the Db2 Analytics Accelerator DTE Site:
https://ibm.biz/Bd2zrp
• What’s available?
• Product videos - for a quick glance at
product features and capabilities
• A guided demo - provides a guided
walk-through of the Accelerator’s
basic capabilities
• A hands-on lab - offers a full product
experience with a guided tutorial
77
Legal Disclaimer
• © IBM Corporation 2019. All Rights Reserved.
• The information contained in this publication is provided for informational purposes only. While efforts were made to verify
the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any
kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject
to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related
to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of,
creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the
applicable license agreement governing the use of IBM software.
• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries
in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at
IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future
product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of,
stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

Db2 Analytics Accelerator Version MDUG May2019

Uploaded by

Copyright:

Available Formats

You might also like

Db2 Analytics Accelerator Version MDUG May2019

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Db2 Analytics Accelerator Version MDUG May2019

Uploaded by

Copyright:

Available Formats

Db2 Analytics Accelerator V7.

• Why Db2 Analytics Accelerator

• Keep data in place for analytics

IBM Db2 Analytics Accelerator for z/OS

Hybrid transaction/Analytical processing

Db2 Analytics Accelerator and Db2 for z/OS

Db2 Analytics Accelerator Version 7.1

Db2 Analytics Accelerator Version 7.1

Db2 Analytics Accelerator Version 7.1

The core of V7.1 Accelerator Deployments

Powered by Db2 with BLU Acceleration

Db2 Analytics Accelerator V7.1, deployment on IBM Integrated

• New generation hardware appliance

IBM Db2 Analytics Accelerator on IIAS

IBM Integrated Analytics System Configurations

M4002-003 M4002-006 M4002-010 M4002-020 M4002-040

Hardware Architecture Overview – Full Rack M4001-010

2x Mellanox 10G Ethernet switches Cooling Requirements:

IBM Integrated Analytics System – Designed for speed

ALL Flash Storage

Designed for Massively Parallel Performance

Db2 Analytics Accelerator Version 7.1, deployment on IBM Z

PR/SM LPAR CPU Memory

• An appliance running on IBM Z

Deployment of an Accelerator Image into an IBM Z Secure

IFLs Dedicated z14 Drawer Storage

Hardware Considerations (contd.)

IBM Db2 Analytics Accelerator on IBM Z

• Data Studio with Db2

Network Connection to Db2 for z/OS

A defined TCP/IP network

Deployment Considerations – Accelerator Instances

Db2 Analytics Accelerator Db2 Analytics Accelerator on Z

On IIAS (e.g., full rack)

Db2 Analytics Accelerator on Z

Db2 Analytics Accelerator on Z

Summary: One API – One implementation – Two deployment options

Out-of-the-box experience Download & go experience

POWER-grade QoS Z-grade QoS

Scalability by scale-out Scalability by using Z: expansion of IFLs and memory

A single Db2 system can connect

Multiple Db2 systems can connect

Deep Db2 for z/OS integration

Db2 for z/OS

Functionality overview (V7.1.5)

• „True HTAP“ – Data coherency between Db2 and Accelerator

• Accelerator-only table usage

Query execution process flow

Accelerator DRDA Requestor

§ Dynamic and static queries can be § Prerequisites for query routing:

SQL functionality support and restrictions

Knowledge Center: Conditions for query routing to an accelerator

Data load and update options with Db2 Analytics Accelerator

Full table load/refresh § Source table data is entirely replaced

Accelerator data load

§ Data is loaded or refreshed using ACCEL_LOAD_TABLES stored procedure

Accelerator data load

Db2 Analytics Accelerator Administrative