What's New in HANA 2.

0 SPS04:
Data Tiering Options

HANA Product Management, SAP

April 17th, 2019

PUBLIC
Disclaimer

The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP.
Except for your obligation to protect confidential information, this presentation is not subject to your license agreement or any other service
or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or any related
document, or to develop or release any functionality mentioned therein.
This presentation, or any related document and SAP's strategy and possible future developments, products and or platforms directions and
functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information in this
presentation is not a commitment, promise or legal obligation to deliver any material, code or functionality. This presentation is provided
without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. This presentation is for informational purposes and may not be incorporated into a contract. SAP
assumes no responsibility for errors or omissions in this presentation, except if such damages were caused by SAP’s intentional or gross
negligence.
All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from
expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates,
and they should not be relied upon in making purchasing decisions.

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

Agenda

Technology Overview

HANA Data Tiering Options

▪ Hot Store
– Persistent Memory
▪ Warm Store
– Native Storage Extension, Extension Node, Dynamic Tiering
– Getting Started
▪ Cold Store
– Spark Controller

Application View on HANA Data Tiering

Data Lifecycle Management

▪ DWF/DLM 2.0 SPS05
© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3
HANA Data Tiering

Solution: Decouple HANA data location from a fixed storage layer

This tier is used to store mission-critical data for real-time processing and
Hot real-time analytics.
Hot Store
Data
Data is retained in-memory of the SAP HANA database.

This tier is used to store data with reduced performance SLAs, which is less
Warm frequently accessed.
Warm Store
Data Data is stored on a lower cost storage tier, managed as a unified part of the
SAP HANA database.

Cold This tier is used to store voluminous data for sporadic or very limited access.
Cold Store Data Data is stored on low cost storage tiers, like disk or Hadoop, managed
separately from the SAP HANA database, but still accessible at any time.

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4

HANA Data Tiering – Technology Overview

Native Storage Extension – NSE

NLS IQ Spark Controller / Hadoop
PMEM Extension Node (BW only)
DRAM SAP IQ
Dynamic Tiering

Hot Store Warm Store Cold Store

Persistent Memory (PMEM) Native Storage Extension (NSE) is an intelligent, SAP HANA cold data tiering provides persistence
extends the in-memory built-in disk extension for the SAP HANA in- capabilities for HANA cold data in external data stores, like
storage capacity for hot data memory database. It is the primary warm store HDFS, Azure Data Lake and SAP Big Data Services.
in HANA. option for HANA on-premise and HANA Service.

Extension node and dynamic tiering will continue

to be offered.

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5

Hot Store
Persistent Memory
 SAP HANA Native Support for Persistent Memory
Officially Supported in SAP HANA 2.3 (April 2018)

Benefit
Larger memory capacity with high Process more data
performance (vs. DRAM & lower tier Persistent Memory Data Reliability
non-volatile faster starts in real-time at a lower TCO with
storage) improved business continuity

Lower TCO data storage hierarchy

Faster start time delivers less downtime > 3 TB

Increased total memory capacity per CPU
Co-innovation with Intel® leads to first Higher Capacity Transforming
fully optimized major DBMS platform than DRAM the memory hierarchy
Early Adoption Program with key
partners/customers ongoing
Intel® Optane™ DC persistent memory available in 1H 2019
12.5x
Improvement in startup time*

First major DBMS vendor to officially support Intel Optane DC persistent memory!
sap.com/persistent-memory

*Internal Benchmark measured with a 6TB dataset in SAP HANA

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7
Memory Configuration
Example: 2 sockets, largest DRAM and largest PMEM configurations

Intel® Xeon™ Intel® Xeon™

Cascade Lake Cascade Lake

128 128 128 128 128 128 128 128

128 128 DRAM 128 128 1,536 GB 128 128 128 128
128 128 128 128 128 128 128 128
3,072 GB 128 128 DRAM 128 128
128 128 128 128
512 512 512 512 128 128 128 128

Persistent
512 512
Memory
512 512 6,144 GB

512 512 512 512

7,680 GB 3,072 GB

Storage Storage
Actual configuration and ratios between DRAM,
PMEM and CPU depend on application sizing.
© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8
Warm Store
Native Storage Extension
Native Storage Extension (NSE) value proposition and use cases
• Value proposition:
• Increase HANA data capacity at low TCO
• Deeply integrated warm data tier, with full HANA functionality
• Will support all HANA data types and data models
• Simple system landscape
• Scalable with good performance
• Supported for both HANA on-premise and HANA-as-a-Service (HaaS)
• Available for any HANA application
• Complements, without replacing, other warm data tiering solutions (extension nodes, dynamic tiering)

• Use cases:
• Any customer built or SAP built HANA application that is challenged by growing data volumes
• S/4HANA data aging (NSE is an evolution of “paged attributes”)
• BW team currently uses extension nodes, but may evaluate NSE in the future

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10

NSE allows the user to specify that certain data is “page loadable”

• Hot “column loadable” data: HANA memory

• HANA is an in-memory database, and loads all data into

Work Space
memory for fast processing.
• Data is “column loadable” and resides completely in Hot data

memory.

• Warm “page loadable” data: Persistence layer

• With NSE, less frequently accessed data may be specified
as “page loadable”.
• “Page loadable” data is loaded into memory in granular HANA memory

units of pages as required for query processing.

Work Space
• NSE will reduce memory footprint for “page loadable”
data. Data is partly in memory, and partly on disk. Hot Warm

• Query performance on warm data may be somewhat

Buffer
reduced compared to hot data. cache

• Data may be converted between “column loadable” and

“page loadable”.
© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC Persistence layer 11
Native Storage Extension (NSE) adds native warm data tier to HANA database
NSE manages “page loadable” warm data in the HANA database with expanded disk capacity, and an intelligent
buffer cache to transfer pages of data between memory and disk.

HANA database HANA database with NSE

Working
area in Working
memory area in
memory

Buffer cache
HANA database Hot data Hot data
capacity = hot in in
data in memory memory HANA database
memory
capacity = hot
data in memory
plus warm data
Persistence layer on disk
Persistence layer

Warm data
on disk

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 12

Specifying data as “page loadable”
• Data may be specified as “page loadable” at table Example DDL:
level, partition level, and column level
• Table:
• Data may be converted between “page loadable” • CREATE COLUMN TABLE T (C1 INT, C2
and “column loadable” VARCHAR(10)) PAGE LOADABLE;
• NSE supports range, range-range, and hash • Partition:
partitioned tables
• CREATE COLUMN TABLE T (C1 INT)
• For hash partitioning the entire table or column must be
page loadable or column loadable
PARTITON BY RANGE (C1) (PARTITION
0 <= VALUES < 10 PAGE LOADABLE,
PARTITION OTHERS COLUMN
Memory P_1 P_2 LOADABLE);
S_A S_B S_C S_D
•
S_E
Column:
Disk P_3 P_4 P_5
• CREATE COLUMN TABLE T (C1 INT, C2
VARCHAR(10) PAGE LOADABLE);
S_F S_G S_H

• Convert table to page loadable:

Partitioned table
• ALTER TABLE T PAGE LOADABLE
IMMEDIATE CASCADE;
© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 13
NSE technical overview
• Column loadable data is fully loaded into
HANA database with NSE memory from disk.
Column store
• Page loadable data is loaded from disk into
Write-optimized delta store
the buffer cache, page by page as needed.
Main store
• Converting column loadable data to page
Pointers to
loadable format moves the data into the
Convert to
page
buffer cache buffer cache.
loadable

Buffer cache
Hot data Warm data • When buffer cache is full, it will eject pages
intelligently based on user access patterns.
Load Periodic
hot write of Load warm
data hot and data • Warm and hot data are written together
warm
data to
from main store to disk during normal
disk savepoint operations.

Persistence layer • The write-optimized store is not paged.

Unified Table
Container

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 14

Tooling
HANA Cockpit:
• Configure buffer cache size (on-premise only;
HaaS will configure this for the user)
• Configure tables, columns, and partitions as
“page loadable”
• Monitor buffer cache usage and capacity
• Report on resident memory status for page
loadable data
• Includes rule-based “recommendation engine” to
monitor user data access patterns.
• Based on statistics, the engine will advise user on
which tables, columns, or partitions would benefit
from being converted to “page loadable”

Data Lifecycle Manager (DLM): Web IDE:

• DLM tool will allow user to convert tables, • Visualized query plan will display when warm
columns, and table partitions between “column data is accessed from NSE in order to satisfy the
loadable” and “page loadable” query
© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 15
 Example: begin with 2TB HANA system
On-premise sizing Database size: 1TB

• HANA system must be scale up (first release) 2TB HANA

Work area
• Determine volume of warm data to add to the (1TB)

HANA database Hot data (1TB)

• May add as much warm storage as desired - up to

1:4 ratio of HANA hot data in memory to warm data
Persistence layer
on disk
• NSE disk store should be no larger than 10TB for Expand HANA database capacity with warm
data that is 4x the size of hot data
first release of NSE

• Divide volume of warm data by 8 – this is size Add HANA memory for buffer cache Allocate buffer cache from
existing HANA memory
of memory buffer cache required to manage 2.5TB HANA
2TB HANA
warm data on disk
Work area (1TB)
Work area (800GB)

• Either add more HANA memory for buffer Hot data Buffer Hot data
(1TB) Buffer
cache, or use some of existing HANA memory cache
(500GB)
OR (800GB)
cache
for buffer cache (will reduce hot data volume) (400GB)

• Work area should be same size as hot data in memory

(equivalent to HANA with no NSE) Persistence layer Persistence layer

4TB warm 3.2TB warm

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 16

Database size: 5TB Database size: 4TB
Roadmap

Initial release Future

HANA scale up only HANA scale out support
Support all data types (including Support remaining data types: graph,
geospatial), except those in the “Future” multi-value (e.g., arrays), time series,
column to the right TEXT
NSE recommendation engine Autonomous tiering of data based on user
access patterns – HANA manages this for
the user
New elastic buffer cache: improved LRU
algorithms, intelligent column prefetch,
integration with HANA execution engine
Page loadable tables, partitions, columns
Tooling: HANA Cockpit, WebIDE, DLM

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 17

Warm Store
Extension Node, Dynamic Tiering
SAP HANA Extension Node – Whats New in SPS04

Common characteristics
HANA Database
▪ HANA node in the scale-out landscape is reserved for warm-data
storage and processing
Scale-Out
▪ Supports all HANA operations and data management features

t Data ▪ Allows larger data footprint of up to 200% of the node DRAM size
▪ HANA persistent memory is supported
Warm Data
Hot Data Extension
Node
New Features:
Worker
▪ Benefits from new partitioning and scale-out features in SPS04:
Node
– range-hash partitioning scheme
– “pinning” tables on fixed HANA nodes
Paged
Warm Data – partition grouping

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 19

SAP HANA Dynamic Tiering – Whats New in SPS04

Feature Description
▪ Addition of TIMESTAMP data type for multistore
tables
▪ Asynchronous table replicas of slowly changing
dimension tables maintained consistently across the SAP HANA System with Dynamic Tiering
HANA and dynamic tiering servers
local join
ATR
T1 T1_REP T2

Benefits DT server
HANA node
▪ 7-digit TIMESTAMP support for multistore tables
rounds out basic data type support in SAP HANA
dynamic tiering
Disk Store
▪ Asynchronous table replicas improves cross-
store query performance by allowing SQL JOIN
operations to be executed close to the data

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 20

Warm Store Options – Getting Started
Customers should start with the built-in Native Storage Extension for warm data. Depending on SLAs, customers may choose extension
node (functionality) or dynamic tiering (data volume) as alternative options. We don’t recommend to mix multiple warm store options in
one landscape, due to complexity reasons.

Costs Functionality

Extension Node

Native Storage Extension

Dynamic Tiering

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC

Data Volume 21
Cold Store
Spark Controller / Hadoop
HANA Spark Controller – Whats New in SPS04

HANA Clients

SAP HANA 2 SPS03

HDP, MapR, CDH, SCP BDS, Azure HDInsight

In-memory SDA HANA Spark Adapter
protocol HANA Spark Controller SPS04
Spark
Hot Data Adapter
Spark Cluster
Cold Data
(HDFS, ADLS)

DLM Views

DLM DLM profiles

New Features in SPS04:

▪ For DLM scenarios, you can now install SAP HANA spark controller on Amazon Elastic MapReduce (EMR)
▪ Support of newer Apache Spark versions: 2.3.x and 2.4.0
▪ Support of newer Hadoop Distributions: CDH 6.1.0, MapR 6.1, HDP 2.6 and HDP 3.0

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 23

Application View
Data Tiering
Which Data Tier Should I Use ?

BW on HANA Suite on HANA

Native HANA
BW/4HANA S/4HANA

In-Memory PMEM PMEM PMEM

Extension Extension
Extension Node Node Node
HANA
Database
Native Storage Extension NSE Data Aging

Extended Store Dynamic

Tiering

BW NLS, ILM Store

Near-Line Storage, SAP IQ BW/4 DTO w/ IQ w/ IQ
External
Store DWF/DLM with BW NLS, ILM/
HDFS, K8s, Cloud Storage Spark Controller BW/4 DTO Archiving

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 25

Data Lifecycle Management
DWF/DLM
DWF/DLM 2.0 SP04 & SP05 – Whats New

▪ SAP DWF 2.0 SP05 is a separate software release, but with

the same shipment date as HANA 2.0 SPS04

▪ SAP DWF includes the Data Lifecycle Management tool

(DLM)

▪ SAP DWF 2.0 SP05 release is based on the XSA software

stack and supports:
▪ Dynamic tiering multistore tables
▪ Generating groups of tables that are common in their data sets
(same columns) and move them together to another storage
location
▪ Defining “nominal key” to enable the relocation of data for tables
without a primary key, which is mandatory to relocate data

▪ SAP DWF/DLM version based on XSC continues to

be supported for SAP IQ

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 27

Thank you
Contact information

Robert Waywell: robert.waywell@sap.com

Andreas Schuster: andreas.schuster@sap.com
Daniel Felsmann (DLM): daniel.felsmann@sap.com
SAP HANA Product Management
AskSAPHANA@sap.com