Enterprise Wide Data Warehousing with SAP BW

SAP NetWeaver Regional Implementation Group -BI SAP AG

Content

Content Enterprise Wide BW - Background Elements of a Corporate BW Strategy The BW Data Model Prevents Silos BW Data Layers Guarantee Reliability and Flexibility BW Corporate Landscape Patterns

SAP AG 2004, Enterprise Wide BW, J. Haupt / 2

Enterprise Wide BW
Background Continuously increasing number of BW installations Steady growth and coverage of BW installations High degree of customer-satisfaction with BW Broad analyst acceptance of BW and the packaged solution approach In general a renaissance of data warehousing

Introduce an enterprise wide BW Strategy and Guidelines that guarantees : Flexibility Reliability at low costs What offers BW to help you realizing an enterprise Data Warehouse Strategy ? What are the elements of an enterprise BW Strategy ?
SAP AG 2004, Enterprise Wide BW, J. Haupt / 3

Enterprise Perspective on BW
Enterprise perspective on BW: Higher level organizational units want to harmonize the lower level BW initiatives Architecture-based BW implementation from the very beginning Stabilize a mature BW landscape Division A Division A (Overhaul the architecture)
...... ......

Group Group
Division B Division B Division C Division C Division D Division D

Retail Retail

Business Business Marketing Marketing

....... .......

Europe Americas Europe Americas

Asia/ Asia/ Pacific Pacific

Germany Germany Spain Spain UK UK France France Nordic Nordic

SAP AG 2004, Enterprise Wide BW, J. Haupt / 4

Multiple BW Implementations Challenges

Source

Source

Unit 1 BW

Consistency issues: Uncontrolled data flows Source Source Multiple extractions of same data Costly development Local BW Not aligned data models Source
Stream C

Danger of high level Silos

Group BW Local BW Stream A BW

Source

Local BW

BW Unit 2 BW

Source

Local BW Local BW Source

Stream B BW Source Source Source

Successful Data Warehousing means: controlled redundancy !

SAP AG 2004, Enterprise Wide BW, J. Haupt / 5

Local BW Implementation Challenges

Observations: Incremental set up Solution-Focus (Data Marts) Departmental view Project is responsible for the entire Data Warehouse Process
Real World

....
BW Project Team

Information requirements SchemaModeling Data MartsInfoCubes

Potential consequences: Departmental islands (silos) Consistency problems Completeness not sufficient Flexibility restricted Trace back not possible

Business Rules Transformation Extraction Sources

Successful Data Warehousing means: controlled redundancy !

SAP AG 2004, Enterprise Wide BW, J. Haupt / 6

Corporate Data Warehousing Strategy

Missing corporate strategy and guidelines
Means: Uncontrolled Redundancy Limited Solution Focus Potential consequences: Silo solutions Doubtful reliability Restricted flexibility High costs

Prerequisites to set up a proper strategy:

Awareness
about accepted data warehousing concepts and their benefits

Support of corporate management (Sponsorship)

If there is no organizational momentum toward a common goal, then the best architecture, the best framework in the world is bound to fail.
W.H. Inmon

SAP AG 2004, Enterprise Wide BW, J. Haupt / 7

Who needs a Corporate BW Strategy? All, but the prerequisites are different!
Group Headquarter Stream A Stream B Stream C Stream D

Region A Region B Region C

Organization, Business
...... Retail Business Marketing ......

countries

Regions/ countries

Necessity of a corporate BW Strategy

Architecture-, SAP-based, single Instance

Legacy

OLTP landscape

Legacy Legacy R/3

Legacy

Legacy R/3

SAP AG 2004, Enterprise Wide BW, J. Haupt / 8

Content

Content Enterprise Wide BW - Background Elements of a Corporate BW Strategy The BW Data Model Prevents Silos BW Data Layers Guarantee Reliability and Flexibility BW Corporate Landscape Patterns

SAP AG 2004, Enterprise Wide BW, J. Haupt / 9

Elements of a Corporate BW Strategy

Think of: Building a Corporate Information Factory * - Avoid Silos
BW Enterprise Data Warehousing Decision Areas
Architecture Data Layer Data Model
Application Development Integration Strategy Support

Landscape

Operational Decision-Making

Tactical/ Strategic Decision-Making

*The Corporate Information Factory: introduced by Inmon, Imhoff

SAP AG 2004, Enterprise Wide BW, J. Haupt / 10

The Role of a Consistent Data Warehouse Data Model

Inconsistent Data Warehouse Data Models Island-solutions (silos, stovepipes) Consistent Data Warehouse Data Model long term success

?
A challenge for Data Warehouse consistency

Material Management OLTP

Sales OLTP

HR OLTP

Not aligned operational Data Models

SAP AG 2004, Enterprise Wide BW, J. Haupt / 11

Consistent corporate Data Model

The BW Data Warehouse Data Model

BW offers as part of the Business Content a predefined expandable Data Warehouse Data Model

BW Data Warehouse Data Model for SAP und other applications (InfoObjects & InfoSources)

Consistent mapping of operative data models

For all SAP applications For SAP industrial solutions
(e.g. Retail, Utilities..)

For non SAP application

Ascential Partnership Siebel, Oracle ....
Legacy Data Models

SAP Corporate Data Model

SAP AG 2004, Enterprise Wide BW, J. Haupt / 12

BW Extended Star Schema built on the BW Data Model

0CUSTOMER 00CUSTGR ...... 0MATERIAL 0MATGR 0MATTYPE ...... 0SALESPERS 0SALESORG .....

0CUSTOMER 0MATERIAL 0AMOUNT

BW InfoObject Master Data: Shared/ Conformed Dimensions

BW InfoCube:

BW Extended Star Schemas built on BW Data Model

scope-specific

BCT InfoSources 0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT Subject Areas
Master Data

BW Data Model defined by Business Content BCT Extractors/ DataSources

MaterialType Materialgroup Sales ORG

Transaction Data

Customer

Material Sales Transaction

Sales Person

Corporate Data Model: SAP Components

SAP AG 2004, Enterprise Wide BW, J. Haupt / 13

BW Extended Star Schemas Prevent Silos

Conformed, Shared Local Part of
Material Dimension
Material Master Table

Part of Material Dimension

Local Part of
Material Dimension

0MATERIAL
Material_Dimension_ID 0MATERIAL Material Dimension Table
Material_Dimension_ID SalesOrg_Dimension_ID Time_Dimension_ID Customer_Dimension_ID Sales Amount Quantity

Material Type

Material_Dimension_ID 0MATERIAL 0PROD_HIER Material Dimension Table

Material_Dimension_ID ..........._Dimension_ID .............Dimension_ID .............Dimension_ID

Material Text Table

0MATERIAL
Language Code Material Name

Material Hierarchy Table

Vertriebsorganisation
Region 1

Region 2

Region 3

InfoCube A

Bezirk 1

Bezirk 2

Bezirk 3

Bezirk 4

Bezirk 5

Gebiet 1 Gebiet 2 Gebiet 3 Gebiet 3a Gebiet 4 Gebiet 5 Gebiet 6 Gebiet 7 Gebiet 8

InfoCube 2 Key figure B

FACT Table

Key figure 1

FACT Table

Material Dimension: local & conformed part

Conformed dimensions enable virtual BW MultiProvider scenarios

SAP AG 2004, Enterprise Wide BW, J. Haupt / 14

BW DWH Data Model and Enterprise Wide BW

The BW Data Warehouse Data Model is the glue, which ties everything together!
It protects against silos (stove pipe solutions, islands)
Within a BW Instance Within a BW Landscape

It is the basis for communication

Drill Thru between BW solution structures (InfoCubes, ODS-Objects) within a BW Instance within distributed BW Instances Drill Thru between BW solution structures and SAP applications

Between people

Enterprise BW Strategy:
Usage of delivered BW DWH data model whenever possible Central controlled guidelines for modifications and expansions

SAP AG 2004, Enterprise Wide BW, J. Haupt / 15

Content

Content Enterprise Wide BW - Background Elements of a Corporate BW Strategy The BW Data Model Prevents Silos BW Data Layers Guarantee Reliability and Flexibility BW Corporate Landscape Patterns

SAP AG 2004, Enterprise Wide BW, J. Haupt / 16

Introducing a BW Layer Architecture to Guarantee Reliability and Flexibility

SAP Business Information Warehouse and the Corporate Information Factory (CIF)*
Data Acquisition
Staging Area Cleansing Transform

Primary Data Management

Data Warehouse

Data Delivery
Architected Data Marts

Ex tracti on / O pen S agi ng t

Finance

any source

Open D is tribu ti on

Master Reference Data

Transaction Data

Logistic

Operational Data Store

The Data Mart is customized and/or summarized data derived from the data warehouse

* Source: Bill Inmon

SAP AG 2004, Enterprise Wide BW, J. Haupt / 17

any target

BW Architected Data Mart Layer: The Departmental Layer What can we expect?
0CUSTOMER master

0CUSTOMER InfoCube A 0MATERIAL

0CUSTOMER InfoCube B

0CUSTOMER InfoCube C 0MATERIAL

BW Extended Star Schema: InfoCubes InfoObject Master Data

(conformed dimensions)

0MATERIAL master

The BW Data Mart Layer based on the BW Data Model and a proper Project Method offers
Consistency within the Data Marts (InfoCubes, ODS-Objekte) Control of data redundancy (MultiProvider, Queries) All desired historical views ...
SAP AG 2004, Enterprise Wide BW, J. Haupt / 18

BW Architected Data Mart Layer: The Departmental Layer Corporate Needs ?

It is not the task of the scope-oriented Data Mart layer to anticipate all kinds of future arising needs this would overload the schemas and corrupt performance It cannot be expected that the data mart project teams have the 360 corporate view

We are limited from a corporate perspective in terms of

Flexibility, Completeness Reliability, Consistency

if we concentrate ourselves just on Data Marts.

BW Data Warehouse Layer the corporate layer

SAP AG 2004, Enterprise Wide BW, J. Haupt / 19

The BW (Enterprise) Data Warehouse Layer: The Corporate Layer

SAP Business Information Warehouse and the Corporate Information Factory (CIF)*
Data Acquisition
Staging Area Cleansing Transform

Primary Data Management

Data Warehouse

Data Deliver
Architected Data Marts
Open D is tribu ti on

Ex tracti on / O pen S agi ng t

Master Reference Data

Finance

any source

Transaction Data

Operational Data Store

SAP AG 2004, Enterprise Wide BW, J. Haupt / 20

Logistic The result of the data transformation and cleansing process is stored persistently: Subject oriented Integrated Granular * Source: Bill Non volatile (historical) Inmon Not scope-specific (not flavored)

any target

The BW (Enterprise) Data Warehouse Layer The Corporate Layer What can we expect?
Reliability, Trace back Prevent Silos
Single point of truth All data have to pass this layer on its path from the source to the summarized edw-managed data marts

Controlled Extraction and Data Staging (transformation, cleansing)

Data are extracted only once and deployed many Merging data that are commonly used together

Flexibility, Reusability, Completeness

The data are not manipulated to please specific project scopes (unflavored) Coverage of unexpected ad hoc requirements (Support the Unknown) The data are not aggregated Old versions are not overwritten or changed but useful information may be added

Integration
Data are integrated (as far as possible) Realization of the corporate data integration strategy

...

The BW Data Warehouse Layer is the corporate memory, the corporate information repository
SAP AG 2004, Enterprise Wide BW, J. Haupt / 21

Introducing the BW Data Warehouse Layer

Ownership
Corporate Guidelines must be defined and administrators must be established to achieve the goals of a Data Warehouse layer Web templates, Reports Queries Business Logic Data Marts
(InfoCubes)

Project Team

1
Set up
Incremental set up in parallel with data marts guarantees buy in from business users

2 2 1

Business Logic

1 1 1 1

(E)DW Layer
(ODS-Objects) (ODS-

(E)DW Admin Team

Technology
BW ODS-Objects Expanded InfoSources define ODS-Object structures BW flexible staging for Master Data
SAP AG 2004, Enterprise Wide BW, J. Haupt / 22

StagingTransformation, Cleansing Extraction Sources

1 Increment

BW and Operational Decision-Making

SAP Business Information Warehouse and the Corporate Information Factory (CIF)*
Data Acquisition
Staging Area Cleansing Transform

Primary Data Management

Data Warehouse

Data Deliver
Architected Data Marts
Open D is tribu ti on

Ex tracti on / O pen S agi ng t

Master Reference Data

Finance

any source

Transaction Data

Logistic

Operational Data Store

* Source: Bill Inmon

SAP AG 2004, Enterprise Wide BW, J. Haupt / 23

any target

Operational, Tactical and Strategic Decision-Making in Traditional BW Implementations Most BW customers do not distinguish between strategic/ tactical and operational decision-making using BW features:

Precalculated aggregates of InfoCubes (aggregation awareness) Drill thru

BW Architected Data Marts

tactical/ strategic on summarized data:

Data Marts

Standard Collections
InfoCube with precalculated aggregates Two InfoCubes with Drill Thru InfoCube and ODS-Object with Drill Thru

operational on detailed, slightly summarized data:

ODS-Object

SAP AG 2004, Enterprise Wide BW, J. Haupt / 24

Flexibility of BW Architected Data Marts: Data Marts & Standard Collections

Focus of traditional BW implementations:

support the known

data mart scenarios InfoCubes

summarized

tactical / strategic decision-making

standard collections
detailed, slightly summarized InfoCubes, ODS-Objects

operational decision-making, flexibility flexibility & reliability

support the unknown

edw base tables granular

complete history ODS-Objects

SAP AG 2004, Enterprise Wide BW, J. Haupt / 25

Limits of Using BW the Traditional Way: Introducing BW Operational Data Stores The traditional usage of BW (with or without dwh-layer), which supports all kinds of decision-making using Architected Data Marts has its limits
If we want to provide event-level or close to event-level operational reporting (near real time data warehousing) and/ or If we are confronted with high data volumes (e.g. Retail)
BW Operational Data Store
I no f Cub e O S D O j ect b G ran l a X u rity

I no f Cub e

Data Marts
G ran l a > X u rity

da ily-w ee l y k DWH O S D O j ect b

DWH
G ran l a >=X u rity

PS A s e e a -> e ent l oa s v rl v d So rce u

G ran l a X u rity

Dedicated ODS-Objects (or InfoCubes) with granular data support operational decision making Directly loaded from PSA Short life cycle of data ( Archiving, Near line Storage) They build a BW Operational Data Store

G ran l a X u rity

SAP AG 2004, Enterprise Wide BW, J. Haupt / 26

Abstract Layer Architecture and the Reality of BW as Packaged Solution Focus of traditional BW implementations:

support the known

data mart scenarios InfoCubes

summarized

tactical / strategic
decision-making

operational standard collections

detailed, slightly summarized InfoCubes, ODS-Objects

decision-making

& flexibility flexibility, reliability & data mining operational

decision-making

support the unknown

edw (base tables) granular

complete history ODS-Objects

support low latency & high volume

SAP AG 2004, Enterprise Wide BW, J. Haupt / 27

Operational Data Stores

BW ods / BW PIPE for retail
others most granular

& data mining

Content

Content Enterprise Wide BW - Background Elements of a Corporate BW Strategy The BW Data Model Prevents Silos BW Data Layers Guarantee Reliability and Flexibility BW Corporate Landscape Patterns

SAP AG 2004, Enterprise Wide BW, J. Haupt / 28

Enterprise Data Warehouse as Single Point of Truth

The Corporate Information Factory*
DSS Applications Departmental Data Marts
Acctg Finance Marketing Sales ERP ERP ERP CRM ETL Staging Area

Changed Data

eComm.

EDW
Global ODS
Oper. Mart

Bus. Int.

ERP Corporate Applications

Exploration warehouse/ data mining

local ODS

Granularity Manager

Cross media Storage Management

Near line Storage

Session Analysis Dialogue Manager Cookie Cognition Preformatted dialogues

Archives
Web Logs

Internet
SAP AG 2004, Enterprise Wide BW, J. Haupt / 29

* Source: Bill Inmon

Basic Landscape Types

The ideal corporate BW landscape that supports all kind of decision-making needs ?
Hub & Spoke BW Enterprise Data Warehouse Enterprise BW
Others Others

Spoke BW Spoke BW Spoke BW

Local

Outside-IN Multiple Hubs & Global BW

BW
Local

BW
Local

Global BW

BW

Select the [IT] approach that fits most closely with corporate Prof. Sethi, University of Texas strategy
Information&Management, 2/01

There is no one size fits all BW landscape!

SAP AG 2004, Enterprise Wide BW, J. Haupt / 30

Some Influencers of a Corporate BW Landscape

More and more operational, low-latency decision-making using BW
operational decision making operational BW Objects tactical/ strategic decision making edw & edw managed data marts

Mission critical BW applications in BW

business critical applications mission critical applications

Local interests
local freedom central governance

Flexibility for time of migration and future acquisitions

SAP AG 2004, Enterprise Wide BW, J. Haupt / 31

Single Corporate Wide BW Instance

Parameters Company time zones Mission critical dominates others Workload from operational reporting Scalability
PSA

External Data Marts BW Architected Data Marts Data Standard Marts Collections latency: >= 1 day mission critical business critical others

Staging Area

BW EDW

BW ODS latency: near real time mission critical business critical

SAP AG 2004, Enterprise Wide BW, J. Haupt / 32

Handling Mission Critical and Operative Decision-Making Separately

Parameters Company time zones Mission critical dominates EDW Migration path from single Instance EDW separated from majority of data marts Parameters Company time zones EDW close to majority of data marts

mission critical strategic

business critical others

BW ODS

mission critical strategic

BW ODS business critical others BW ODS BW EDW

BW EDW

SAP AG 2004, Enterprise Wide BW, J. Haupt / 33

BW ODS

Multiple Central Managed Local BWs to Support Business Critical Scenarios

Asia Local BW business critical others Europe Local BW business critical others Americas Local BW business critical others

ODS

Global BW

Parameters Scalable Mission critical dominates EDW EDW separated from majority of data marts
SAP AG 2004, Enterprise Wide BW, J. Haupt / 34

mission critical strategic EDW ODS

ODS

ODS

Summary

BW offers a wide range of features that support enterprise wide data warehousing needs. Nevertheless a strategy and an organizational momentum must exist to realize an enterprise wide BW strategy.

SAP AG 2004, Enterprise Wide BW, J. Haupt / 35

Questions

Any Questions?

SAP AG 2004, Enterprise Wide BW, J. Haupt / 36

Further Information
Public Web:
www.sap.com > Solutions > SAP NetWeaver

SAPNet:
Use ALIAS: /BW

SAP Service Marketplace:

www.service.sap.com/bw
BW InfoIndex Data Modelling BW InfoIndex Enterprise Data Warehousing BW InfoIndex ODS Functions EDW Whitepaper : http://service.sap.com/~sapidb/011000358700003703 852003E/EnterpriseDWOverviewEN.pdf

www.service.sap.com/education
PDEBW1 Enterprise Wide Data Warehousing with SAP BW (Workshop - 2 days)
SAP AG 2004, Enterprise Wide BW, J. Haupt / 37

Copyright 2004 SAP AG. All Rights Reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, WINDOWS, NT, EXCEL, Word, PowerPoint and SQL Server are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, Informix and Informix Dynamic ServerTM are trademarks of IBM Corporation in USA and/or other countries. ORACLE is a registered trademark of ORACLE Corporation. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, the Citrix logo, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, MultiWin and other Citrix product names referenced herein are trademarks of Citrix Systems, Inc. HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C, World Wide Web Consortium, Massachusetts Institute of Technology. JAVA is a registered trademark of Sun Microsystems, Inc. JAVASCRIPT is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. MarketSet and Enterprise Buyer are jointly owned trademarks of SAP AG and Commerce One. SAP, SAP Logo, R/2, R/3, mySAP, mySAP.com and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are trademarks of their respective companies.

SAP AG 2004, Enterprise Wide BW, J. Haupt / 38