Enterprise Data Warehousing With Sap BW - Overview

Enterprise Data Warehousing with SAP BW An Overview
White Paper Version 2.0 August 18th, 2003
juergen.haupt@sap.com
SAP (SAP AG and SAP America, Inc.) assumes no responsibility for errors or omissions in these materials. These materials are provided as is without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party web pages.
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW
V2.0
Table of Contents
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW ....................................... 1 1 INTRODUCTION........................................................................................................................... 1 1.1 1.2 2 3 SUPPORTED SOFTWARE VERSIONS ........................................................................................... 1 THIS WHITE PAPER .................................................................................................................. 1
AIMS OF THE WHITE PAPER ..................................................................................................... 1 MOTIVATION ................................................................................................................................ 2 3.1 3.2 THE MARKET ............................................................................................................................ 2 WHY DO YOU NEED A CORPORATE BW STRATEGY?................................................................. 3
4 5
ENTERPRISE DATA WAREHOUSING WITH SAP BW.............................................................. 5 BW ENTERPRISE ARCHITECTURE........................................................................................... 6 5.1 ASPECTS OF A DATA WAREHOUSE ARCHITECTURE .................................................................... 6 5.2 DATA STORE ARCHITECTURE - BW DATA LAYER ..................................................................... 10 5.2.1 BW Architected Data Mart Layer .................................................................................. 11 5.2.2 BW Data Warehouse Layer .......................................................................................... 14 5.2.3 BW Extraction and Staging........................................................................................... 19 5.2.4 BW Support for Operational / Real Time Reporting ..................................................... 20 5.3 DATA ARCHITECTURE - BW DATA MODEL ............................................................................... 24 5.3.1 Operative Data Models and Data Warehouse Data Model .......................................... 24 5.3.2 The BW Data Model ..................................................................................................... 25 5.4 TOPOLOGIES BW LANDSCAPES ........................................................................................... 29 5.4.1 Consistency in a distributed BW Landscape ................................................................ 30 5.4.2 BW Landscapes und Technical Parameters ................................................................ 31 5.4.3 Inside-Out Landscape Architecture - BW as Central Enterprise Data Warehouse...... 32 5.4.4 Outside-In Landscape Architecture - Decentralized BW Data Warehouses................ 34
6 7
CENTRAL BW APPLICATION DEVELOPMENT ...................................................................... 38 DATA AND DATA MODEL INTEGRATION .............................................................................. 40 7.1 7.2 7.3 CHALLENGES POSED BY DATA INTEGRATION............................................................................ 40 DATA MODEL INTEGRATION ..................................................................................................... 42 THE ROLE OF SAP MASTER DATA MANAGEMENTS (MDM) ...................................................... 43
SUMMARY .................................................................................................................................. 45
TABLE OF ILLUSTRATIONS 43
2003 SAP AG AND SAP AMERICA, INC.
CONTENT
BW Enterprise Data Warehousing Overview
V2.0
1 Introduction
1.1 Supported Software Versions
This document is relevant for BW Versions 3.x.
1.2
This White Paper
The subject of this document is very complex. Therefore, we do not claim to cover all the various aspects of this topic, or to offer an exhaustive description of the areas discussed. To see updated versions of this document, visit the SAP Service Marketplace regularly.
2 Aims of the White Paper

This White Paper offers an overview of the implementation of SAP Business Information Warehouse (SAP BW) from a corporate perspective. It considers the following issues: Architectural aspects: data management, data models, topologies etc. on a conceptual level Integration aspects: supporting corporate strategy as far as the harmonization of master data is concerned Solution development: the efficient, consistent development of BW-based applications.
Enterprises differ in terms of their organization and their business. This implies that there can be no standard solution for a corporate BW implementation strategy. However there are basic truths that always have to be considered, and patterns that arise within specific business types. Above all, this White Paper will describe fundamental aspects of a corporate-wide BW implementation. This White Paper is no substitute for a business-specific consultation on BW architecture. This White Paper focuses on the general principles involved in a strategic corporate BW implementation and not on exceptions to this. It follows the 80-20 rule. The White Paper avoids details wherever possible, so as not to lose sight of the overall context. Knowledge about BW is useful to read this White Paper but no prerequisite for it.
V2.0
3 Motivation
3.1 The Market
At the time of writing, there are more than 6000 BW installations within the market. The fundamental advantages of BW have led more and more customers to center their corporate data warehousing strategy around BW. Customers frequently stress the end-to-end conception of the BW in this context. In comparison to fragmented technologies, the integrated metadata concept, from data integration right through to analysis, leads to lower total costs for the project overall. The BW has moved away from isolated instances to incorporate an architecturally-based view. In this way, the BW has come to set new standards of quality in terms of its positioning within an enterprise. The following graphic shows various rudiments of this strategy:
Evolution of SAP BW
Isolated BW Implementations
BW1 BW2
Strategic Corporate BW Implementations

Headquarter Reporting Scenario
BW1
Global BW
BWn
Others
BW2
Local
BW BW BW
BW3
Local
Global BW
Architected BW Landscape Scenario
BWn Issues: Synergy Integration Consistency
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 3 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 3
Local
Enterprise BW
Spoke BW Spoke BW
BW Enterprise Data Warehouse Scenario
Illustration 1: Evolution of the SAP BW This leads to one main motivation for this paper. It aims to support the evolution-process of BW offering an overview of the essential criteria for corporate architected BW implementations for customers and consultants. On the other hand, there is still a large number of customers who do not have a corporate data warehouse strategy and who have implemented BW in a more isolated and restricted way. Thus, another motive is the hope of making people aware of the prospects and benefits of a business-wide, architecturally-based BW implementation.
V2.0
3.2
Why Do You Need A Corporate BW Strategy?
There are many training guides, enhanced documentations, ASAP Accelerators and How to Guides available that offer support to BW users. These documents all focus on offering support with concrete challenges that arise during a project. This is obviously important and necessary. However there are also short-comings as projects are always part of an incremental BW implementation: From project to project, project goals are modeled and realized in BW. These goals are typically reporting- and analysis-driven, and the overall success of the project is then often measured by decision-makers in terms of how far these reporting and analysis demands are met. As such, the focus is at solution level or using a data warehouse term the focus is at data mart level. Less attention is accorded to decisive factors for the mid- and long-term success of an investment in BW: Redundancy has to be controlled in all areas! Business-wide guidelines rarely exist regarding: The persistent BW data stores and their design The BW Data Warehouse data model and its management The BW landscape (the role of BW instances and which conditions are valid for new instances). Aspects of data integration are often neglected too, and applications in various organization units that feature a BW are developed asynchronously and redundantly. Altogether, from an overall business perspective, you are left with a costly and less efficient procedure that delivers neither consistent, reliable, nor integrated information. The following graphic clarifies the solution-focus issues:
Redundancy and Data Mart (Solution) Focus

Real World

Impacts of non-existing corporate BW guidelines: Redundant Data Redundant Extraction Redundant Transformation Redundant Business Rules Redundant Masterdata ......
Information .... Requirements

(grouped to Scopes)
SuccessfulData DataWarehousing Warehousingmeans means Controlled ControlledRedundancy Redundancy!! Successful
BW Application Project Team
Schema-Modeling InfoCubes/ ODS-Objects
Business Rules Transformation Extraction Sources
Illustration 2: Redundancy and Solution focus
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW The next graphic clarifies the multiple BW landscape issues:
V2.0
Redundancy and Multiple BW Landscape

Global
e ap sc am Ex e pl
HQ BW SubDiv BW
Impacts of non-existing MDMS corporate BW guidelines: Redundant Data Flows Redundant Extraction Redundant Development Redundant Models ......
BW Asia BW N-America BW S-America
BW Local
L al Re if
W eB
nd La
BW Sales Europe
Sub-Division A R/3 Global
Sub-Division B R/3 only Germany
Sales Europe R/3
Div Div Div .................... R/3 R/3 R/3
12 systems worldwide
Source Systems
SuccessfulData DataWarehousing Warehousingmeans meansControlled ControlledRedundancy Redundancy !! Successful

SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 5
SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 5
Illustration 3: Redundancy and multiple BW landscape
The more complex an enterprises structures, the more important it is to have corporate-wide guidelines and recommendations for the implementation of BW. They guarantee long-term consistency and reduced costs.
V2.0
4 Enterprise Data Warehousing with SAP BW

When we talk about enterprise data warehousing it is first necessary to ask what we understand by this term: Enterprise data warehousing is the sum of all the decisions that have to be taken at corporate level in order to obtain a durable solution that adheres to business-wide, specified guidelines, and fulfills all requirements for integrated, consistent and structured information. These decisions are mirrored in the enterprise data warehouse architecture. In this chapter we will consider the aspects of such an architecture so that we can then look more closely at how BW supports an architecturally-based, consistent, corporate data warehouse solution. As well as a BW architecture that ensures consistency, we want to consider in what ways the BW offers support in generating consistent application solutions. Furthermore, in discussing a corporate BW strategy, we have to include BW in the data integration strategy of the company. This is done in the last chapter. As with all important decisions, we concentrate on the fundamental aspects, following the 80:20 rule.
V2.0
5 BW Enterprise Architecture
An architecture can be seen as a system design decision that has a specified duration. In other words, once it has been implemented it is not easy to change. In defining a corporate BW architecture it is clear that the term architecture is used in various senses, and that aspects such as where, what and how often become confused. This leads to confusion and is dangerous in that essential factors may be considered from a lop-sided perspective, or not at all.
5.1
Aspects of a Data Warehouse Architecture
In discussing an optimal BW architecture, an enterprise often focuses too quickly on physical BW instances. Is more than one BW instance needed and if yes, where are they to be institutionalized? How will they work together? These questions concern a BW landscape, or as we prefer to call it, a consistent enterprise BW topology. This means that the second or even third architecture elements are being considered before the first. How can individual BW instances and their tasks be discussed without having first defined how data is consistently handled and saved in such a BW instance? We must start with the BW data store or data layer architecture: The layer architecture defines persistent data stores and their storage schemas from extraction through to the end-user, and refinement processes between the layers. Qualified data status and the respective layers: Staging Area - row Data Warehouse Layer - integrated, granular, not application-specific, historical Data Mart Layer - integrated, aggregated, application-specific Operational Data Store (ODS) integrated, granular, application-specific, close to event

Conceptual Layers of Data Warehousing
V2.0
Non-SAP Source Persistent Staging Area SAP Source
Operational Data Store
Data Warehouse
Architected Data Marts
Information Access
Archiving & Near-Line Storage
Illustration 4: The conceptual layer architecture of data warehousing Repeatedly saving raw data in various refinement situations means on the one hand side redundancy. On the other hand side means a consistent layer architecture the only way to control the redundancy of data warehousing, which derives from its very nature. This has to be taken into consideration with the layer data store schemas. However, a suitable layer support is not the only necessary condition for a corporate-wide data warehouse architecture that can secure consistency in the mid- and long-term. This can only be achieved if the operative data models are mapped consistently to the model of the data warehouse layer and the model of the data mart layer. Such a consistent data (model) architecture therefore describes the model of each layer and how the models relate to each other. As such we need a data architecture that guarantees, for example, that operative material terms, as they are found in the respective sales or distribution area, or in materials management, are reproduced consistently in the material subject area. Ideally you would have a single operative enterprise data model and use that to derive the model for each layer. However, a corporate data model of this sort does not normally exist, particularly if legacy systems are being used. This means that often no or only limited guidelines and rules exist how to map the operative models to a single data warehouse layer model and a single data mart layer model because of the difficulties to achieve this. The result is unsynchronized isolated solutions (stovepipe solutions). This is illustrated in the following graphic:
V2.0
Operational Data Models and the Data Warehouse Data Model

Inconsistent data warehouse data models stovepipe solutions Consistent data warehouse data model long term success
A data warehouse consistency challenge
Not aligned operational data models
Consistent enterprise data model
Illustration 4: Operational Data Models and DWH Data Model A missing data model architecture is one reason why corporate data warehouse strategies fail. Only when there is clarity regarding the data layer and data model does it make sense to discuss the corporate-wide topological aspects of a BW architecture. Because topological aspects are influenced by the layer architecture. Two basic topologies are differentiated: The BW enterprise data warehouse landscape with BW as the central information hub Landscape based on local BWs with a higher level global BW
InsideInside-Out BW Enterprise Data Warehouse Enterprise BW
Others
Others
Spoke BW Spoke BW Spoke BW
Local
OutsideOutside-IN BW Landscape Global BW
BW BW BW
Local
Local
Illustration 5: BW basic topologies When determining the BW topology, political, organizational and technical factors play an important role. As a result you cannot really talk about a generally valid optimal BW topology. We summarize that it is necessary to make definitions that are valid enterprise-wide in the following areas:
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW BW data layer architecture Which data statuses does it make sense to save persistently? Which designs are approved for the various data stores?
V2.0
BW data model architecture Each data warehouse layer requires a data model that describes objects and their relationships The data models of the various layers have to be derived from each other consistently Different corporate cultures, geographical, and business diversifications lead to different data warehouse landscapes.
BW topology architecture
V2.0
5.2
Data Store Architecture - BW Data Layer
It is generally accepted that the data store that supports analysis and reporting is to be kept separate from the data store that deals with any sort of data acquisition. Analysis and reporting are handled by data marts. Close to event-level operative reporting is handled by the operational data store. BW supports multidimensional data marts in an extended star schema. This is made up of BW InfoCubes and over-arching BW InfoObject master data dimensions. Operational data stores are supported by flat BW ODS objects. Here too BW master data acts as structures that span ODS objects. Data acquisition (processes that secure quality and integration) are referred to as staging processes. The persistent saving of staging process data occurs in staging areas. Data staging stores are realized by the Persistent Staging Area (PSA) Objects and by ODS objects. The granular, integrated result of the staging process reproduces data in optimal condition. This is referred to as the data warehouse layer. BW ODS objects are the central storage objects that build a BW data warehouse layer. This gives us the following general picture of a BW layer architecture:
SAP Business Information Warehouse

Data Acquisition Primary Data Management
Data Warehouse
Master Reference Data
Staging Area
Cleansing Transform
Data Delivery
Architected Data Marts Extended Star Schemas

Open Distribution
Extraction / Open Staging
Finance
any source
Transaction Data
Logistic
ODS
Flow Control
Meta Data
Illustration 5: BW layer architecture - overview
any target
10
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW 5.2.1 BW Architected Data Mart Layer
V2.0
InfoCube data marts allow the flexible generation of queries and navigation to integrated, granular or aggregated data. In order to make comparisons possible, an amount of historical data (defined by you) can be retained. InfoCubes are multidimensional data structures (also called star schemas) that organize data in such a way that key figures (or facts or measures) are affixed to characteristics and their attributes, which form the so-called dimensions (BW InfoCube dimensions and master data). To increase the capacity of an InfoCube schema, dimensions are separated into local and global operative parts. Global, because these dimension-parts are only saved once and are conjointly addressed by various InfoCubes.
Local Part of
Material Dimension
Material Dimension
Material Master Table Material Number Material Number Material Type Material_Dimension_ID Material Number Material Dimension Table
Material Text Table Material Number Material Number Language Code Language Code Material Name Material Hierarchy Table
Vertriebsorganisation
InfoCube
Material_Dimension_ID SalesOrg_Dimension_ID Time_Dimension_ID Customer_Dimension_ID Sales Amount Quantity FACT Table
Region 1
Region 2
Region 3
Bezirk 1
Bezirk 2 Material Group Bezirk 3 Bezirk 4 Bezirk 5
Gebiet 1 Gebiet 2 Gebiet 3 Gebiet 3a Gebiet 4 Gebiet 5 Gebiet 6 Gebiet 7 Gebiet 8
Shared, global BW Extended Star Schema

Part of Material Dimension
Illustration 5: The BW extended star schema In this way BW InfoCubes support the concept of conformed dimensions, which form a sort of glue between InfoCubes.
Horizontal Consistency: BW Architected Data Mart Layer
CUSTOMER master CUSTOMER InfoCube MATERIAL MATERIAL master CUSTOMER InfoCube CUSTOMER InfoCube MATERIAL Horizontal Consistency: Conformed Structures BW InfoObjects master data
Illustration 6: Horizontal consistency in the BW architected data mart layer
11
V2.0
Conformed dimensions essentially support horizontal consistency within the data mart layer. They are part of the BW concept and help prevent the occurrence of isolated solutions (stovepipes). The BW term for these conformed dimensions is master data. This is one reason to qualify the BW data marts as Architected data marts. The effectiveness of the BW InfoCube schema does not just support consistency but is obviously also essential for mapping end-users demands. Above all we have to mention here the ability to map entities from different historical views to the same key figure. By using surrogate keys in the InfoCube and because of the master data/ conformed dimension concept, the BW makes it possible to simultaneously map different views to key figures in one InfoCube.
BW Support of Different Historical Views

Constellation 09/1998
Customer Mother-Company AAA BBB CCC Customer Date AAA BBB CCC BBB CCC 09/1998 09/1998 09/1998 10/1998 10/1998 X X Y Revenue 100 100 100 100 100 Report using todays (10/98) constellation Mother-Comp Rev 09/98 Rev 10/98 X Y 100 200 0 200
BW supported Views in a single InfoCube Extended Star Schema
Report using 09/98 constellation Mother-Comp Rev 09/98 Rev 10/98 X Y 200 100 100 100
Fact Table
AAA BBB CCC
Report showing historical truth Mother-Comp Rev 09/98 Rev 10/98 X Y 200 100 0 200
Customer Mother-Company X Y (changed) Y
Report showing comparable results Mother-Comp Rev 09/98 Rev 10/98 X Y 100 100 0 100
Constellation 10/1998
SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 6 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 6
Illustration 7: BW and slowly changing dimensions 'Today is Yesterday' : Show key figure values, including historical key figure values, from the currently valid perspectives (current view of dimension attributes) 'Yesterday is Today' : Show key figure values from user-defined perspectives (key date oriented view of dimension attributes) 'Historical Truth' : Show key figures (transactions) in the same state (historicization of dimension attributes), as at the time of posting historically correct
Scenarios can also be presented in which only key figure values with unchanged dimension attribute constellations are reported. In order to reduce redundancy and counteract the one report is equal to one InfoCube error, virtual multidimensional structures are supported for persistent InfoCubes. These are called BW MultiProviders and BW Queries.
12
V2.0
Multi-dimensional Schemas in BW
EndEnd-User Reporting & Analysis need BW BEX Report
end-user requirement specific
MultiMulti-dimensional model
BW InfoCube(s)
physical implementation basic facts not report specific
BW BEX Query
virtual Structures complex KPIs navigation behavior report (type) specific
BW MultiProvider
virtual Structures basic facts not report specific optional
Illustration 8: Persistent and virtual multidimensional structures in BW Despite the capabilities of BW data mart layers, the limitations and dangers of a data mart-focused implementation are also evident: Limited reusability of data and loss of historical conditions Data is dealt with in the data mart layer in a scope-specific manner: Aggregation/ manipulation/ overwriting of data is normal. It follows that the usage of these data for other purposes is limited. The data mart layer is the direct access layer for the end-user. Performance, therefore, plays a decisive role. For this reason only data that is actually needed is retained in the data mart layer. Saving data just in case will quickly lead to a large data volume and impact negatively on performance. Overwriting old images, as normally happens in the conformed dimensions (master data), is therefore valid and indeed desired. However, this means that history is lost.
Redundant data storage (the same data is held in various InfoCubes and often in different degrees of granularity) Redundant data acquisition (staging) Redundant data extraction Limited ability to trace data. Data staging is required by the BW layer architecture here. Factors arising from contact with the data mart data model that threaten consistency are discussed in the chapter on data model architecture.
13
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW 5.2.2 BW Data Warehouse Layer
V2.0
In order to overcome the limitations and dangers of a data mart-focused BW implementation, the corporate BW architecture has to answer demands such as: extract once deploy many single point of truth Furthermore, the contradiction between quite sensibly reducing information in the data mart layer if it is not currently required by the end-user, and the claims the BW makes to flexibility, has to be resolved. The BW also has to be able to respond quickly to new requests. The answer is to build a separate layer that saves the integrated results of the data transformation and data quality-ensuring processes in a consistent, granular and historical form: The BW data warehouse layer. Ideally this is built centrally and extends business-wide. We can then talk about a BW enterprise data warehouse (layer).
A Matter of Consistency and Reliability The BW Data Warehouse (Layer)

Data Warehouse
Staging Area
Cleansing Transform
Data Delivery

Open Distribution
Finance
any source
Transaction Data
Logistic
ODS
Flow Control
Meta Data
Illustration 9: SAP BW data warehouse layer The following description of a BW data warehouse layer is based on William H. Inmons definition: A BW data warehouse layer offers: o o o o Subject-oriented, integrated data Original data that will not be changed in relation to the goals of a particular project (unflavored data) Granular data Non-aggregated data Historical data (that cannot be overwritten or deleted)
any target
14
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW o A single point of truth
V2.0
The BW data warehouse layer is the only source for BW architected data marts. (extract once deploy many). In this way a BW data warehouse offers the following benefits: I. Flexibility o o New InfoCube data mart solutions that include history and new extraction can be built in real-time
The BW can respond quickly to unpredictable, one-off user requests, without having to build persistent structures (Although the BW data warehouse layer is not to be misused for standard analysis as it does not possess the optimal structures for this). II. Reliability and Reproducibility o The BW data warehouse layer is centrally administrated and not subjected to arbitrary projects. The data warehouse layer is the common source for all the data provided in the data marts.
III. Complete history IV. Consistency o o o Extraction, staging and integration are managed centrally Data is extracted only once Redundant staging processes are avoided.
A BW data warehouse (layer) is an enterprises memory, a repository for information for the whole enterprise. In addition to offering horizontal consistency within the BW data mart layer, the BW data warehouse layer offers a sort of vertical consistency that the data mart layer alone is not able to guarantee.
15

Vertical Consistency: BW Data Warehouse Layer
CUSTOMER master CUSTOMER
V2.0
CUSTOMER
CUSTOMER
InfoCube
MATERIAL
InfoCube
InfoCube
MATERIAL
Horizontal Consistency: Conformed Structures BW InfoObjects master data
MATERIAL master
Vertical Consistency: Single Point of Truth BW DWH Layer ODSODS-Objects
Transaction / Master Data
Source 1
Source 2
Source 3
Source 4
Source 5
Illustration 10: BW data warehouse layer and vertical consistency The question of how to organize data warehouse layers often causes arguments. Extreme positions on this range from 3NF to de-normalized star schemas. We do not want to give a definite answer here, but suggest taking a pragmatic approach. As the data warehouse layer forms the basis for supplying the data marts, data should be saved so that it can be easily transported into conformed dimensions and InfoCubes. This means complex join operations for normalized data warehouse objects on the way to the data marts should be avoided. It is therefore, for example, legitimate to store position data and header information for a sales order together in the data warehouse layer. To the question, "what kind of database design is appropriate for a data warehouse?", W.H. Inmon offers the following answer: For a data warehouse, it is always better to have a lightly normalized design. Star schemas fit elsewhere (s. Data Marts t. author). The data warehouse design is not perfectly normalized because a few alterations are made for common usage, such as creating arrays of data when data is always used together, creating a small and selective amount of redundancy where appropriate, merging tables that are commonly used together, and so forth. At the end of the day, a data warehouse design is distinctively normalized. This answer presumes that you know the difference between a data warehouse and a data mart. Restatement or regeneration of some of these higher levels of information requires that we keep available a detailed historical data store. This should be clean (free of errors and conformed to the Data Warehouse master data model) to avoid unnecessary reprocessing of source data. (W. H. Inmon, Architecture 2002)
16
V2.0
The BW does not offer a data store specific for the data warehouse layer. Transaction data is saved in flat ODS objects, master data, reference data, in InfoObjects or ODS objects. Handling history is supported in BW by staging processes (transfer/ update rules) and staging ODS objects.
Data Mart Master Data Table Customer A B DWH Object Customer A B A Source S1 S1 S1 Activ From 20020101 20020101 20020401 Activ To 20020331 99991231 99991231 CustomerGroup X Z Y after 2nd Load: 20020401 CustomerGroup Y Z
after 2nd Load: 20020401
Sourcesystem S1
Customer A B A
CustomerGroup X Z Y
1st Load: 20020101 2nd Load: 20020401
Illustration 11: Example - BW data warehouse: Historization of master data With master data, the source system normally does not deliver versioned or historical data images. Furthermore full-loads are often provided for master data. If the source system does not provide any historization, the BW staging processes must provide this information. The management of activity time slots is supported by BW for data warehouse InfoObjects automatically. For data warehouse ODS objects activity time slots must be maintained during staging. It is also the task of the BW with full loads to determine the changed records. This is done using special staging ODS objects. These tasks are omitted when SAP Master Data Management (SAP MDM) historicized Delta images are provided. Handling transaction data is easier, as long as the transaction is not to be posted again. With changeable transactions a simple time stamp is often not enough to prevent it from being overwritten.
17
V2.0
Customer Dimension A
InfoCube
Material Dimension 4712 4713 Time 200211 200211 Amount Company Currency 100 150 Time Dimension 200211
Customer
Material 4712 4713
BW Architected Data Mart Layer

other InfoCube
A A
other InfoCube
monthly
BW Data Warehouse Layer
Customer A A A Time 20021107-10am 20021107-3pm 20021107-4pm
weekly
DocNo 1 1 2 Pos 10 10 10 Material 4711 4712 4713 Amount Local Currency 100 200 300
daily
Amount Company Currency 50 100 150
daily
Sourcesystem
Customer A A A Time 20021107-10am 20021107-3pm 20021107-4pm DocNo 1 1 2 Pos 10 10 10 Material 4711 4712 4713 Amount Local Currency New booking 100 Correction booking 200 New booking 300
Illustration 12: Example - BW data warehouse: Transaction data The quantity of data in a BW data warehouse layer mounts quickly. The BW/ SAP Archiving Development Kid (ADK) provides the prerequisites for the data warehouse layer to cope with the demand, without the memory space used escalating exorbitantly. If a large data volume is to be made available online, the BW supports the usage of nearline storage.
18
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW 5.2.3 BW Extraction and Staging
V2.0
In order to guarantee the quality of data in the BW data warehouse layer, corporate-wide rules with respect to extraction and staging are important. First and foremost, these should avoid redundant extraction and transformation. Using BW as the basis for enterprise data warehousing requires that all data sources can be addressed. For this reason the BW offers a wide spectrum of support for extraction: Predefined, enhanceable extractors for practically all SAP application data The possibility to generate extractors for customer-specific SAP applications like COPA Generic extractors for customer-specific enhancements of mySAP applications Direct extraction from databases supported by SAP (BW DB-Connect Feature), based on table- or view-definitions Flat file extraction Integration with Ascentials Datastage BW 3.5 it is scheduled to support extraction from sources that can be addressed using JDBC or ODBO . Most extractors for SAP application transaction data are delta-enabled. This means that transactions can be written to a delta queue at the time of posting. They are then extracted from this delta queue into the BW. It is evident that as well as delta-enabling, important design possibilities for BW scenarios are also derived from this, as master data information like price, for example, can also be saved at the time of posting. Extracted data is the first status at which it is relevant to save data persistently. This raw data is saved in the persistent staging area (PSA) on a 1:1 basis. The PSA forms a backup for newly designed staging processes.
Staging Cleansing Area: Transform PSA
BW Data Warehouse
any source
BW ODS
Illustration 13: BW staging Whether and which data statuses between raw and integrated data is saved persistently has to be decided on a case by case basis. This decision depends on the quality of data and the outlay for potential data harmonization. BW ODS objects are used as data stores.
19
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW 5.2.4 BW Support for Operational / Real Time Reporting
V2.0
Staging, data warehouse and data mart layers constitute classic data warehousing. Classic in the sense that data is subjected to an exact and predefined procedure in order to guarantee the quality and integrity of reporting and analysis. Classic data warehousing also means the ability to take the burden away from the operative systems.
The classic data warehousing approach:

Dedicated load and staging processes High quality Optimized retrieval structures Reduce workload on OLTP systems
Master Reference Data Transaction Data
Data Mart Finance Data Mart Logistic
Illustration 14: Classic data warehousing Alongside the classic layers (staging, data warehouse and data mart) an additional data area is often postulated: The Operational Data Store (ODS). The operational data store supports operative reporting, whereas data marts are data structures for tactical and strategic reporting and analysis. In practice, if operative reporting does not demand real-time data, the physical division between the data mart and the ODS becomes artificial. Mostly loading data two or three times daily is sufficient for operative reporting. Updating data on a daily basis may even be enough. The greater the latency (the time difference between the operative event and the acquisition of this event for reporting purposes), the more sensible it is to store granular data in data mart InfoCubes too, and to use this both for tactical analysis and operative reporting. Flat BW ODS objects are better suited to purely listing data. In this way, most BW implementations already support operative reporting and so relieve the pressure on operative systems.
Tactical / Strategic
Data Warehouse
Architected Data Marts
20
V2.0
In most of the cases operational reporting needs can be supported in BW using the classic data warehouse process
Architected Data Marts/ ODSODS-Objects
Data Mart Finance Data Mart Logistic ODS-Object Sales Orders
Data Warehouse
Master Reference Data Transaction Data
..most of the cases..: classical data warehousing has its limits if we want to provide event-level or close to event-level operational reporting (near real time reporting) and/ or if we are confronted with huge data volumes
Illustration 15: Classic data warehousing supports operatives reporting BW supports this tactical analysis and operational reporting on the same InfoCube through precalculated aggregates, aggregation awareness in the OLAP Engine, and query catching. However it is to be pointed out that the utilization profile of data for operative usage is different in regard to both its history and its granularity to data for analytical usage. With large volumes of transaction data this has to be accounted for in the schema modeling. With delayed operative reporting, which follows the path of classic data warehousing, there is no need to define a dedicated BW operational data store. You do then have to be aware of the possible consequences of basing analysis and operative reporting on the same structures. One the other hand, if data is requested close to the event time in the operative system, the classic pull principle of the BW data warehousing boundaries are set. In this case we talk about real-time or near real time operational reporting. The data extracted (in real time) is written straight to ODS objects, without going through the data warehouse layer. These objects form the BW operational data store.
Tactical / Strategic / Operational
21
V2.0
Near Real Time Data Warehousing (Apollo) BW Operational Data Store

Data Warehouse
Staging Area
Cleansing Transform
Data Delivery

Open Distribution
Finance
any source
Transaction Data
Logistic
ODS
Flow Control
Meta Data
Illustration 16: BW operational data store: Near real time data warehousing Data from an operational data store is not to be processed directly in the data mart layer for reasons of data consistency. If data from an operational data store is needed in a data mart scenario, you should always go through the (enterprise) data warehouse. To summarize, the functionalities planned and offered by BW to support real time reporting scenarios are: BW Remote InfoCubes external InfoCube fact tables BW allows you to define InfoCubes whose fact tables are themselves not persistent in the BW. The data is then read by an extractor during the query runtime and subjected to normal InfoSource treatment as far as transfer and update rules are concerned. It can then be made available to the OLAP processor. All data sources that can be addressed by BW can make external fact tables available in this way. BW Virtual InfoCubes a program that behaves like a fact table BW near real time data warehousing (next main BW release) With the next main BW release it will be possible to push data into BW ODS objects directly after either its emergence or the change event. Direct reporting to operative structures (next main BW release) Reporting directly to operative structures will also be an option if integration with other data is not required. However the burden produced (lack of data integration and data quality controls) means tight limitations are set for this option.
any target
22
V2.0
BW and Operational/ Real Time Reporting

Key-Questions: Data Latency ? Data Integration ? Workload on OLTP Systems ? Ul Apollo Apollo
BW Near real time BW remote data warehousing InfoCube BW virtual InfoCube
BW
BW classic data warehousing
Adaptors
BW Data Integration BW Data Acquisition
JDBC ODBO XMLA SAP Query
OLTP
Illustration 17: BW and operational / real time reporting
23
V2.0
5.3
Data Architecture - BW Data Model
The BW layer architecture describes the life-cycle of data from the event through to the retrievalrelevant data mart or flat ODS object structure. One thing is intuitively clear: persistent stores and their storage schemata alone are not sufficient to ensure controlled redundancy. This requires a data warehouse data model. 5.3.1 Operative Data Models and Data Warehouse Data Model
The operative applications entity relationship model serves as the basis for a data warehouse data model. Ideal examples of an business-wide, operational data model are scarce. The challenge then is to derive a consistent data warehouse data model from a multitude of operative models. The following graphic illustrates this:
Inconsistent data warehouse data models stovepipe solutions Consistent data warehouse data model long term success
A data warehouse consistency challenge
Not aligned operational data models
Consistent enterprise data model
Illustration 18: Operational data model and data warehouse data model Generating a business-wide data warehouse data model based on operative data models, which are not harmonized, is a complex and political procedure. This is why the data warehouse data model architecture is often neglected. The result is isolated applications or stovepipe data marts that are not integrated. In this regard BW customers finds themselves in a good position. BW offers an extensible data warehouse data model that represents the operative models of the SAP applications and a large number of industry-specific solutions consistently as part of the BW Business Content. Data models also exist for specific applications from the competitive environment (Oracle Financials, for example). BW customers with a high operative degree of coverage from SAP applications can enjoy being in the situation illustrated in the following graphic:
24

B W C o n s is te n t d a ta w a re h o u s e d a ta m o d e l fo r S A P A p p lic a tio n s a n d o th e rs lo n g te rm s u c c e s s
V2.0
A d a ta w a re h o u s e c o n s is te n c y c h a lle n g e
N o t a lig n e d o p e ra tio n a l d a ta m o d e ls
S A P e n te rp ris e d a ta m o d e l
Illustration 19: Data warehouse data models and the situation for BW customers The BW Business Content data model watches over the incremental generation of InfoCube solutions and ensures that they match, often without the customer even having to be aware of this. This is the true value of BW Business Content from an architecture point of view, and one reason for the success of BW. 5.3.2 The BW Data Model
Operative entities and attributes correspond to BW InfoObjects.
0MATTYPE 0CUSTOMER
0AMOUNT 0SALESORG
BW Data Model
0MATGR 0DOCNO
0MATERIAL
0SALESPERS
operative entities and attributes BW InfoObjects

Enterprise Data Model: e.g. mySAP Component Object Models
Entities, Attributes -> BW InfoObjects

MaterialType Materialgroup Sales ORG
Customer
Material Sales Transaction
Sales Person
Illustration 20: Enterprise data model and enhanceable BW data model: InfoObjects
25
V2.0
Alongside technical properties like format and length, InfoObjects carry data warehousing-relevant properties: Aggregation behavior (key figures) Standard displays (key figures) Textual description Language-dependant descriptions External hierarchies Clustering operative entities and attributes to subject areas takes place through InfoSources. Master data and transaction data InfoSources are differentiated:
Clustering operative entities and attributes Data Warehouse subject-areas BW InfoSources built of InfoObjects
BW BW Data Model definedby by Data Model defined BusinessContent Content Business
BCT InfoSources Subject Areas
0MATERIAL 0MATGR 0MATTYPE
Master Data
0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT

Transaction Data
MaterialType
Materialgroup
Sales ORG
Customer
Material
Sales Person
Sales Transaction
EnterpriseData DataModel: Model: Enterprise e.g. mySAP Component e.g. mySAP Component ObjectModels Models Object
Illustration 21: BW data model: InfoSources and subject areas InfoSources always represent a set of InfoObjects and describe relations between these. InfoSources are the central bridge between the operative and the BW data model. In the operative system they correspond in form to a DataSource that, in turn, represents an extractors result structure. In the BW data mart layer an InfoSource either defines a conformed dimension (master data InfoSources), or a relation between conformed dimensions (transaction data InfoSources), in other words, the basis for InfoCube schemata.
The following graphic illustrates the role of BW InfoSources with regard to the data mart layer data model:
26
V2.0
BWExtended ExtendedStar StarSchemas Schemas BW

0CUSTOMER 00CUSTGR ......
0MATERIAL 0MATGR 0MATTYPE ......
0SALESPERS 0SALESORG .....
Shared/Conformed ConformedDimensions Dimensions Shared/

BWData DataMart MartLayer Layer BW Data Model defined by Data Model defined by Business Content Business Content
0MATERIAL 0MATGR 0MATTYPE
Master Data
InfoCubeswith withLocal LocalDimensions Dimensions InfoCubes
0CUSTOMER 0MATERIAL 0AMOUNT
0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT

Transaction Data
BCT Extractors/ DataSource

Materialgroup
MaterialType
Sales ORG
Customer
Sales Person
Illustration 22: BW data model: data mart layer data model In the BW (enterprise) data warehouse a master data InfoSource and additional time-slice or time-stamp attributes defines a data warehouse ODS object, which is able to store different versions of a subject area, which occur over time. Time-slices or time-stamps for master data are normally not offered by the source systems. This information has to be provided in the transfer or update rules during staging. As a rule of thumb a transactional-InfoSources define the structure of a data warehouse ODS object for transactional data (see following graphic):
27
V2.0
EDW InfoObject/ ODS-Object
ODS-Object
+ Insert Timestamp/ Activity Time Slice + Source
BWData DataWarehouse WarehouseLayer Layer BW Data Model defined by Data Model defined by BusinessContent Content Business
If Overwrite: Unique Identifier +Source
0MATERIAL 0MATGR 0MATTYPE Master Data
0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT Transaction Data
BCT Extractors/ DataSource

Material group Sales ORG
Material Type
Customer
Sales Person
Illustration 23: BW data model: Data warehouse layer data model BW Business Content offers a pre-defined extensible data model for SAP components based on InfoObjects and InfoSources. This data model describes the BW data mart layer and also extends the BW data warehouse layer to include time slices. If the Business Content data model for mySAP components is used throughout, data mart InfoCube scenarios for a particular themed area can be built without producing isolated applications.
28
V2.0
5.4
Topologies BW Landscapes
The short-term need to optimize already existing BW landscapes The parallel implementation of BW and R/3 Or simply a short-term focus on factors pertaining to costs.
The topology aspects of a BW architecture often interest people most. Some reasons for this are:
However, the discussion in the previous chapters has made it clear that focusing on landscape aspects alone is superficial and in no way productive if strategies regarding the persistent layer and its data models are not also developed. If this is now clear, we can pose the question: What does the ideal BW landscape for my enterprise look like?
The Ideal Corporate BW Landscape ?

Strategic Corporate Implementation Options:
InsideInside-Out BW Enterprise Data Warehouse Enterprise BW
Others Others
Spoke BW Spoke BW Spoke BW
Local
OutsideOutside-IN BW Landscape Global BW
BW BW BW
Local
Local
Illustration 24: The ideal corporate BW landscape? Unfortunately we have to disappoint you: There is no one size fits all BW landscape Naturally there are solutions that are to be favored in terms of consistence, but enterprise-specific circumstances often prevent such solutions. Decisions on BW landscape architecture and ultimately on all aspects of the architecture have to be made amidst the following conflicts:
29
V2.0
Enterprise-wide Strategies
Centralized versus Decentralized
Low dispersal, high centralization High dispersal, low centralization High dispersal, high centralization
Inside-Out versus Outside-In

Central Data Hub, Local Spokes Local Hubs, central aggregation
Select the [IT] approach that fits most closely with corporate strategy
Prof. Sethi, University of Texas Information&Management, 2/01
Global Integrations versus Local Responsiveness

Global scale Global standardization Global demand Local customer needs Local market conditions Local governmental regulations
Illustration 25: Enterprise-wide strategies and BW architecture Further parameters relevant to this decision are: Master data integration status and strategy To what extent is an awareness of the essential roles of consistent data available? Budget Sponsorship IT Strategy - Operative System Landscape Competitive environment.
As a basic principle the following advice should be followed: Select the [IT] approach that fits most closely with corporate strategy (Prof. Sethi, University of Texas Information&Management, 2/01) In other words, a business-wide BW enterprise data warehousing strategy will hardly ever be successful when it contradicts the corporate culture. 5.4.1 Consistency in a distributed BW Landscape
The corporate BW landscape will often consist of several BW instances and possibly also external data warehouse tools. Alongside general decisions that ensure consistency within and between BW layers, rules must also be defined to ensure the consistency of data and metadata in a distributed data warehouse landscape. BW supports distributed (BW) landscapes with a multitude of functionalities: Data consistency The distribution of data in a landscape like this has to be controlled (delta handling). This is guaranteed by BW functionalities:
30
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW o o Data Mart Interface (BW-BW data distribution) Open Hub Service (BW-Third Party data distribution)
V2.0
Metadata consistency This is realized and controlled using the BW development environment and the BW metadata interface (XML for 3rd-party) Daily Operations Partnerships with producers of automization tools for data centers guarantee that the data distribution process (BW process chains) can be controlled and scheduled from one central point. These functionalities are the same whatever BW landscape architecture you choose. 5.4.2 BW Landscapes und Technical Parameters
Diverse technical parameters also influence what BW landscape is suitable for your enterprise. Geographical distribution of users In the most extreme cases users can be spread around the whole world. This graphic illustrates challenges that then arise:
Availability and Maintenance

Availability and Maintenance
24 x 7 Access Performance impacts Data actuality Data backup, repair and recovery System upgrades and maintenance Zero Downtime
B ru sse ls B ru sse ls 1 2 a m 1 2 a m 2 a m 2 a m 4 a m 4 a m 6 a m 6 a m 8 a m 8 a m 1 0 a m 1 0 a m 1 2 p m 1 2 p m 2 p m 2 p m 4 p m 4 p m 6 p m 6 p m 8 p m 8 p m 1 0 p m 1 0 p m L .A . L .A . 3 p m 3 p m 5 p m 5 p m 7 p m 7 p m 9 p m 9 p m 1 1 p m 1 1 p m 1 a m 1 a m 3 a m 3 a m 5 a m 5 a m 7 a m 7 a m 9 a m 9 a m 1 1 a m 1 1 a m 1 p m 1 p m T o k y o T o k y o 8 a m 8 a m 1 0 a m 1 0 a m 1 2 p m 1 2 p m 2 p m 2 p m 4 p m 4 p m 6 p m 6 p m 8 p m 8 p m 1 0 p m 1 0 p m 1 2 a m 1 2 a m 2 a m 2 a m 4 a m 4 a m 6 a m 6 a m R e p o r tin g e p TR im e o r ti n g T im e
B B rru us ss se e lls s F 4 F rr 4p pm m 8 8p pm m S Sa a 1 12 2a am m 4 4a am m 8 8a am m 1 12 2p pm m 4 4p pm m 8 8p pm m S Su u 1 12 2a am m 4 4a am m 8 8a am m 1 12 2p pm m 4 4p pm m 8 8p pm m M Mo o 1 12 2a am m 4 4a am m 8 8a am m 1 12 2p pm m
R Re ep po o rrttiin ng g
Illustration 26: BW landscape and availability / maintenance Code pages in different languages have to be supported BW is Unicode-compatible. Concrete solution scenarios have to be agreed with BW product management.
31
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW 5.4.3
V2.0
Inside-Out Landscape Architecture - BW as Central Enterprise Data Warehouse
It is indisputable that business-wide, consistent information can soonest be guaranteed where there is a central BW instance offering data warehouse layer services. This means the business-wide consolidation of all relevant data in a granular, integrated, non-volatile, unflavored and subjectoriented form. William H. Inmon forwards this solution in the corporate information factory.
BW as Central Enterprise Data Warehouse (EDW)

BW Enterprise Data Warehouse: BW Data Warehouse layer as single point of truth for the entire corporation
Enterprise BW
Spoke BW Spoke BW
Illustration 27: BW as Corporate Information Factory All data that acts as the basis for tactical and strategic data marts has to go through the enterprise data warehouse. Extraction straight into data marts, ignoring the enterprise data warehouse, is not permitted. Exceptions have to be controlled. There must be a strategy in place for using these data marts as the basis for operative reporting, as well as for using the enterprise data warehouse itself for operative reporting. A landscape of this sort with the inherent demands single point of truth and extract once, deploy many offers the ideal prerequisites for controlling redundancy at corporate level. It is often assumed that this solution, although ideal, is practically unrealizable. Without going into the arguments in detail, it is sufficient to say that this is not necessarily the case for BW customers.
32
V2.0
Many large companies that operate globally make a large investment in order to unify their operative landscapes by implementing SAP R/3. As a result, the ideal prerequisites for a central enterprise data warehouse approach emerge: integration of data (common coding), integration of data models into the SAP R/3 data model, the building of an operative R/3 landscape, which takes the company organizational and technical conditions into account. In other words, a centralized approach in the operative environment leads directly to a solution option for the enterprise data warehouse topology. This type of ideal environment is often found in single division enterprises that find themselves in strongly competitive environments and as a result, have a high sense of awareness for the usage of more reliable and efficient IT solutions. This awareness mostly involves a high degree of support from the highest management level for a BW enterprise data warehouse solution.
Inside-Out: Central BW Instance Covers All

Inside-Out: Central EDW BW Instance Covers All
External Data Marts PSA BW EDW integrated non-volatile no flavor
Enterprise BW
Spoke BW Spoke BW
BW Architected Data Marts tactical/ strategic reporting
Staging Area
operational like reporting
less granular
aggregated
granular
BW ODS operational/ near real time reporting
granular
Illustration 28: Scalability of a BW Corporate Information Factory I A BW enterprise data warehouse based landscape will be implemented incrementally for cost reasons. Often, first a BW instance will be built that offers data warehouse layer as well as data mart layer services, including operative-level reporting. Obviously the aim of the enterprise data warehousing strategy also has to be to source all external data marts with data from the BW enterprise data warehouse only! It is then possible to separate reporting and analysis services as the load increases.
33
V2.0
Spoke BW Spoke BW BW Enterprise Data Warehouse Scenario
Inside-Out: Central EDW BW Instance and BW Spoke Instances

Inside-Out: Central EDW BW Instance as Information HUB Architected Data Marts Spoke Instances
External Data Marts
Enterprise BW

PSA Staging Area BW EDW integrated consistent
PSA
BW ODS operational/ near real time reporting BW O DS granula r operational reporting BW ODS Architected BW granular Data Marts operational reporting granular tactical/
less granular
Corporate BW Data Mart strategic reporting
tactical/
aggregation
aggregated
granular
Illustration 29: Scalability of a BW Corporate Information Factory II To summarize, the conditions for a central BW enterprise data warehouse solution are always most favorable within a single division company with strong headquarter. 5.4.4 Outside-In Landscape Architecture - Decentralized BW Data Warehouses
Even where it is preferable to have a single BW enterprise data warehouse for the whole enterprise, the complexity of the different business areas in a company that operates globally can lead to shared BW landscapes. On this issue William H. Inmon says: Simply building a central single data warehouse does not address the size and complexity of the problem posed by the need for integrated, historical easily accessible data across a complex global enterprise.
W.H. Inmon Managing Multiple Data Warehouse Development
Often political circumstances or a companys lack of an over-arching concept lead to shared BW landscapes too, even when the business conditions seem to be favorable. Enterprises that choose diversification should generally check that the outlay involved in integrating data on a granular level, as required with the enterprise data warehouse approach, is not greater than the benefit. Even when the ideal prerequisites exist such as for a single division enterprise, certain conditions (a regional enterprise structure, for example, without process integration between the regions), coupled with technical and political factors, can lead to an enterprise moving away from the central BW enterprise data warehouse concept. This leads to a landscape that comprises several local BWs. Local here means that one of these BWs covers only one part of the organization. The description of what local can mean in individual cases depends on the size and the complexity of the company.
34
V2.0
Each local BW should operate like an enterprise data warehouse for its own area. In this way, the sum of the local enterprise data warehouses forms a quasi virtual enterprise data warehouse. Unlike the central enterprise data warehouse that requires all corporate data to be integrated at granular data level, this is not necessary with local enterprise data warehouses, although it is still preferable. Each local enterprise data warehouse necessarily ensures a local granular data integration. Having a landscape architecture based on local BWs means that for over-arching reporting and analysis, at least one additional integration layer is required to consolidate the data from the local BWs. A BW that consolidates the data from other BWs and offers this in an integrated form is called a global BW. Again, the term global, and the scope of a particular global BW can only be defined on an individual, company-specific basis.
Architected Multiple BW Landscape

Global
Global BW
Headquarter
Illustration 30: Local - Global BW Landscape In terms of an enterprises data integration strategy, it is important that master data harmonization takes place on an aggregated level (global BW)1. In these terms a global BW of this sort is always an integrator.
Divisional
BW Europe Data Marts Data Warehouse ODS
Global BW Division A
BW Americas Data Marts Data Warehouse Staging ODS
BW Asia Data Marts Data Warehouse Staging ODS
Regional Local
1
Staging
virtual BW Enterprise Staging Data Warehouse
R/3-ERP Europe I
mySAP CRM
R/3-ERP Americas I
R/3-ERP Americas II
R/3-ERP Asia
Integration is then only demanded at the higher hierarchy levels of a subject area or dimension. This is generally easier to achieve than integration at granular level. As a result, a topology of this kind often mirrors the integration strategy of an enterprise that does not choose granular common coding.
35
V2.0
The situation of local BWs or legacy systems with respect to harmonization determines the integration efforts at global BW site: 1. Local BWs are to be integrated I. It is the task of the global BW to gather data from local BWs (Data values are already common coded, local-global data models are integrated, a data model architecture exists) II. It is the task of the global BW to consolidate and harmonize data (mapping) (Data values are not common coded, local-global data models are integrated, data model architecture exists) III. It is the task of the global BW to consolidate and harmonize data (mapping) and to integrate the local-global data models. (A data model architecture either does not exist, or is not controlled. Statements about model consistency cannot be made without doing individual checks.) 2. Local BWs and legacy systems are to be integrated I. For legacy systems data and data model integration is necessary. These scenarios copy the different corporate data warehousing strategies. The essential difference lies in whether local and global BWs are developed according to common architecture guidelines (scenario 1.I and 1.II). If this is the case we can talk about an architected outside-in BW landscape. This harmonized, architecture-based interaction of local and global BW instances is a serious option for building a BW based corporate data warehousing strategy from the very beginning. It is supported by the central development of BW templates for local reporting and analysis.
Architected Multiple BW Landscape: BW Template Roll-Out

Global
Global BW
Headquarter
Divisional
BW Europe Data Marts Data Warehouse Staging ODS
Global BW Division A
BW template roll-out
BW Asia Data Marts Data Warehouse Staging ODS
BW Americas Data Marts Data Warehouse Staging ODS
Regional Local
virtual BW Enterprise Staging Data Warehouse
R/3-ERP Europe I
mySAP CRM
R/3-ERP Americas I
R/3-ERP Americas II
R/3-ERP Asia
R/3 template roll-out
Illustration 31: Architected Multiple BW Landscape and BW Template roll-out
36
V2.0
However, if dealing with a mature BW landscape, you will probably echo scenario 1.III and scenario 2. There is obviously only a restricted corporate strategy for local BWs and it is the task of the global BW to take the first step towards a corporate strategy (catch the BWs scenario).
Focus on Integration and Headquarter Reporting Global BW as the Corporate Integrator

Global Global
BW BW
BW BW
local local
BW BW
local local
others others
Legacy/ Files
ERP/ Legacy
ERP/ Legacy
Illustration 32: Global BW as corporate Integrator
37
V2.0
6 Central BW Application Development

As well as the need for consistent information on all organizational levels, avoiding redundant application development (data marts) is also a motivation for a business-wide BW strategy. Redundant application development poses a constant threat to data consistency. Redundant application development is often accompanied by a certain arbitrariness in the management of persistent data stores and BW data models. The general solution approach is to develop applications or templates for all the organization units of an enterprise centrally. To what extent applications are generated on a local level (local-local applications) and/ or corporate templates are adaptable locally, is always company-specific. It is symptomatic that this decision is influenced more by political than functional factors. Two extreme positions stand out regarding the local management of templates: Stable templates No local changes to the central template are permitted Example templates The centrally produced template acts as an example only. The first is self-evident: The fewer changes are allowed to the central template, the easier the central further development and maintenance of these templates, and the better it is for overall consistency in the BW. The description of this scenario sounds very rigid. This is not entirely true as it can work very well if local power users are allowed to generate BW queries for template InfoProviders and MultiProviders2 within this scenario. It is then possible to define, for example, local virtual multidimensional and flat structures, local KPIs, selections, and layouts. In the stable template scenario, the template is transported in the form of the active version of the template metadata (A versions). If a local application development is intended in addition to central application development, this normally means that local development systems exist. If a central template is changed in this environment, importing a new version of the central template in the A version (see above) will overwrite local adaptations. As of BW Version 3.5 the generation of customer content is possible to support this scenario. This enables you to generate and transport meta objects in a D version, as is the case with Business Content. An import into the local target system does not affect the existent active meta objects. Meta objects are merged with existing objects only when the D version is activated. When there are conflicts, how to proceed must be resolved locally.
InfoProvider: InfoCube, ODS objects; MultiProvider: Union of InfoCubes and/ or ODS objects. 38
V2.0
Content Delivery at the Customer Site

Customer Content System (Headquarter)
M(odified) A(ctive)
1 1
1 1 1 1 1 1
2 2 1 1 1 1 Export (D Object)
create Subsidiary
activate change
import M(odified) A(ctive) D(elivery) 1 1 1 1 1 1 1 1 4 4 1 1 1 1

change
Content activation
Illustration 33: BW Customer Content If locally adapting the central template is permitted an additional type of persistent data store is added locally: distilled data. These are data stores (usually ODS objects) whose structure and method of filling cannot be changed locally as they serve to prepare data for over-arching global reporting. If it can be changed locally, the central template can no longer fill these roles. The different structures of a local BW instance illustrates the following picture.
source for corporate reporting
standardized corporate defined local reporting
distilled
corporate
local defined local reporting
Local BW
Data Structures
Illustration 34: BW Customer Content
local
39
V2.0
7 Data and Data Model Integration

Integration is the central issue from a corporate point of view. In the data warehouse area this primarily concerns the integration of data from different sources and the integration of data models from different operative components. Data integration in BW directly addresses the topic Master Data Management and the SAP solution MDM (Master Data Management).
7.1
Challenges Posed by Data Integration
Both the data warehouse layer and the data mart layer make integrated data available by definition. But data integration in corporate terms is easier said than done. As such integrating data from different sources is often the greatest challenge3. The harmonization of data is therefore not a fact of enterprise data warehousing scenarios, but rather the aim. Even when all operative systems deliver common coded data, you always have to anticipate that data that cannot be integrated may also be delivered in the future because of company acquisition. The result of this is that data that cannot be integrated during load time has to be saved in the BW. Recognizing this in good time is essential. The status of the master data has to be investigated and the enterprise-wide data harmonization strategy has to be considered carefully. This will allow later data harmonization without having to make adjustments. If data cannot be integrated then the different sets of data have to be separated so that they do not overwrite each other.
Not Common Coded Source-Keys as BW-keys *

>> Introducing new Sources later
BW-KEY 0MATERIAL M1 S2 BW-KEY 0MATERIAL M1 S2 Conformed Dimension/ Text InfoObject Master Data PIPES CONSULTING
Date Quantity 20020601 100 20020601 200 InfoCube Fact-table
M1 S2
20020601 20020601
100 200
M1 S2
PIPES CONSULTING
M1 S22
PUMP CONSULTING Master Data Later Introduction of Sourcesystem
Transaction Data
Master Data
R/3-1
* the technical BW implementation is more sophisticated - the key issue exist anyhow
R/3-2
With restricted BW implementations data harmonization is often not a central issue. You may therefore only come up against the data harmonization task later when the BW is implemented as an enterprise-wide data warehouse solution. 2003 SAP AG AND SAP AMERICA, INC. 40
ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW Illustration 35: Source system key as BW key?
V2.0
There are several ways in which this situation can be controlled with BW. For one, by using name spaces different data models can be maintained in BW for the different operative systems that cannot be integrated. Data stores are then separated automatically too. This approach is not recommended for an enterprise, however, it is an option for application centers (which offer BW services for different enterprises). As separating data using different data models is only advisable in odd cases, data should actually be separated in data structures defined by a common data model. This is done by extending the key structure of the BW InfoObject affected using a separator. In the most straightforward of cases this separator will contain origin information (source systems). BW supports separation through concatenation and compounding.
Concatenation
BW-Key ConcatenatedKey U1004711 U2004711 Build Key U1 Separator Source-Value 004711 Source-System 1 Separator U1 U2 Text AUDI BMW
Build Key U2
AUDI
004711 Source-System 2
BMW
Illustration 36: Key value concatenation Concatenation means the data model is not affected. The key values are simply extended by the separator. This can happen centrally in the BW using general transfer rules. Here we have a data separation option that can be influenced by the user and which it is possible to modify at a later date. With compounding on the other hand, the key column of the InfoObjects affected are extended to a source system column. This can be done per InfoObject and is automatically supplied by the BW. Separation is handled by the BW and the database. However this has the disadvantage that it cannot be removed again later (for example, after subsequent integration). We favor concatenation as compounding has further functional limitations when handling complex heterogeneous integration scenarios and is inflexible because of automatism. Using InfoObject master data navigation attributes (conformed dimension attributes) in queries, instead of concatenated InfoObjects, makes realignment to the common coded values after a migration possible. Adjustments to queries are then obsolete. This is illustrated in the next graphic:
41

Power of BW Navigational Attributes
ConcatenatedKey 0MATERIAL S1004711 S4004711 00000123 00000456 Amount 100 200 300 400 Query access Separator S1 S4 00 00 Group-ID IMATERIAL 00000123 00000456 00000123 00000456 Text AUDI BMW AUDI BMW
V2.0
InfoCube Fact Table Date 0MATERIAL 20020930 S1004711 20020930 S4004711 00000123 20021001 00000456 20021001 entries before migration entires after migration
realigned material numbers after migration Query results after migration Text AUDI BMW AUDI BMW Text AUDI BMW 0MATERIAL S1004711 S4004711 00000123 00000456 IMATERIAL 00000123 00000456 2002 100 200 300 400 2002 400 600
Illustration 37: Navigation attributes support the flexibility of BW solutions
7.2
Data Model Integration
Third party and legacy applications, and even different SAP components such as R/3 and EBP (Enterprise Buyer Professional), use different semanitics for, for example, product terms. For this reason there are different subject areas for products in BW (InfoObjects). These various product views are consolidated again by consolidated InfoObjects.
Cross-Component Integration: Product

Global and consolidated views on products, independent from source component
0PRODUCT
0MATERIAL
0SERVICE
0BBP_PROD
0CRM_PROD
Component restricted Analytics
IS: Material
IS: Service
IS: BBP Prod
IS: CRM Prod DataSources Source Components individual data models
R/3 Material
R/3 Service
EBP Product
CRM Product
R/3
R/3 R/3
R/3
R/3 R/3
R/3
EBP R/3
R/3
CRM R/3
SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 44
Illustration 38: Model integration with consolidated InfoObjects
42
V2.0
This obviously does not resolve the challenge of making a common coded key available for the consolidated view 0PRODUCT. SAP MDM functionalities offer support in this respect.
7.3
The Role of SAP Master Data Managements (MDM)
Corporate master data management is a topical issue. SAP offers SAP MDM (Master Data Management).4 Active master data management, the central management and distribution of master data, will not be examined any more closely here. If a central master data management exists, BW is a potential client and the overall corporate reporting and analysis is a beneficiary. More interesting are the possibilities offered by MDM in complex, business-wide BW scenarios without harmonized data. Master data from different source systems can be analyzed for commonalities and a grouping (group ID) corresponding to common coding suggested. This passive functionality is termed content consolidation.
mySAP MDM Master Data Consolidation
Content Consolidation MDM MDM

Client SAP
Core Process Steps: 1. Loading identifying attributes of master data objects as they were maintained in their local applications (clients)
Client non SAP
Load
Client non SAP
?
Matching
2. Matching of uploaded master data objects to identify duplicates
Client SAP
=
ID Mapping
Client non SAP

Description of a master data object: Identifying Attributes Other application specific data SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 45
3. Providing ID Mapping Information based on matching results for subsequent usage (I.e. in Business Intelligence)
Illustration 39: SAP MDM master data consolidation The mapping information (group ID) is then a navigation attribute of the InfoObject to be harmonized in BW (see Illustration 37: Navigation attributes support the flexibility of BW solutions). This can also be added afterwards without interfering with the existing schemata.
Integration between BW and MDM will be offered as of Release BW 3.5.
43
V2.0
SAP MDM supports the model consolidation scenario described in 7.2 (Data Model Integration) through master data harmonization by means of consolidated InfoObjects for an analysis that spans components. Keys of different InfoObjects are checked in MDM for commonalities, mapped to a group ID, and maintained centrally as attributes that span InfoObjects.
Master Data Harmonization
MDM MDM
Client SAP
Master data harmonization

Core Process Steps: 1. Central Creation of master data objects just covering the global information of the master data object 2. Local Creation within the master data environment of the application system (client) and distributing their global information to MDM 3. Continuous matching processes identify duplicates and result in ID Mapping between them 4. Distribution of global master data information to the various clients 5. Local Completion of master data within the local environment 6. Use of ID Mapping information for MDM Analytics like business-wide analyses etc.
+
Client non SAP
Local Creation Central Creation

1.234.237
Local Completion
=? +
Client SAP
6.674.288 634.237 4.002.531 737.108
Matching & ID Mapping
13.282.401
Distribution
Client non SAP
Analytics
Description of a master data object: Globally relevant Data (Client) specific Data SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 46
Illustration 40: SAP MDM master data harmonization As well as active master data management, MDM supports key consolidation and complex master data harmonization through deferred (asynchronous) consolidation. The flexibility of the conformed dimension in BW allows the extended star schemas to be subsequently enhanced with information from the MDM, without interfering with existing solutions.
44
V2.0
8 Summary
Important elements of a corporate BW strategy are architecture, consistent application development and data integration concepts. A corporate BW architecture is essential to mid- and long-term success. An architecture of this kind manifests itself in design and procedural guidelines that are valid enterprise-wide, and in their control. BW supports various aspects of the architecture through a multitude of functionalities that secure consistency. It is up to each individual enterprise to determine how and to what extent these functionalities are used. A central development of BW application templates is an adequate way to support a corporate wide BW architecture and harmonization. Integration of data is an ongoing challenge. BW is tightly integrated with SAPs MDM and offers in addition various design options to meet integration requirements. This has to be considered in a BW strategy. Coming to the end we want to repeat once more the main statement of this paper: The top priority of any corporate BW strategy must always be to control redundancy. This is true for each individual BW, and for the interaction of all the BWs within an enterprise!
45
V2.0
Table of Illustrations
Illustration 1: Evolution of the SAP BW ................................................................................................ 2 Illustration 2: Redundancy and Solution focus ..................................................................................... 3 Illustration 2: Redundancy and multiple BW landscape ....................................................................... 4 Illustration 3: The conceptual layer architecture of data warehousing ................................................. 7 Illustration 4: The BW extended star schema .................................................................................... 11 Illustration 5: Horizontal consistency in the BW architected data mart layer ..................................... 11 Illustration 6: BW and slowly changing dimensions ........................................................................... 12 Illustration 7: Persistent and virtual multidimensional structures in BW............................................. 13 Illustration 8: SAP BW data warehouse layer..................................................................................... 14 Illustration 9: BW data warehouse layer and vertical consistency ..................................................... 16 Illustration 10: Example - BW data warehouse: Historization of master data .................................... 17 Illustration 11: Example - BW data warehouse: Transaction data ..................................................... 18 Illustration 12: BW staging.................................................................................................................. 19 Illustration 13: Classic data warehousing ........................................................................................... 20 Illustration 14: Classic data warehousing supports operatives reporting........................................... 21 Illustration 15: BW operational data store: Near real time data warehousing.................................... 22 Illustration 16: BW and operational / real time reporting .................................................................... 23 Illustration 17: Operational data model and data warehouse data model......................................... 24 Illustration 18: Data warehouse data models and the situation for BW customers............................ 25 Illustration 19: Enterprise data model and enhanceable BW data model: InfoObjects...................... 25 Illustration 20: BW data model: InfoSources and subject areas ....................................................... 26 Illustration 21: BW data model: data mart layer data model ............................................................. 27 Illustration 22: BW data model: Data warehouse layer data model .................................................. 28 Illustration 23: The ideal corporate BW landscape?........................................................................... 29 Illustration 24: Enterprise-wide strategies and BW architecture ........................................................ 30 Illustration 25: BW landscape and availability / maintenance ............................................................ 31 Illustration 26: BW as Corporate Information Factory ........................................................................ 32 Illustration 27: Scalability of a BW Corporate Information Factory I................................................... 33 Illustration 28: Scalability of a BW Corporate Information Factory II.................................................. 34 Illustration 29: Local - Global BW Landscape................................................................................... 35 Illustration 30: Architected Multiple BW Landscape and BW Template roll-out................................. 36 Illustration 31: Global BW as corporate Integrator ............................................................................. 37 Illustration 32: BW Customer Content ................................................................................................ 39 Illustration 33: BW Customer Content ................................................................................................ 39 Illustration 34: Source system key as BW key? ................................................................................. 41
46
V2.0
Illustration 35: Key value concatenation............................................................................................. 41 Illustration 36: Navigation attributes support the flexibility of BW solutions ....................................... 42 Illustration 37: Model integration with consolidated InfoObjects ........................................................ 42 Illustration 38: SAP MDM master data consolidation ......................................................................... 43 Illustration 39: SAP MDM master data harmonization ....................................................................... 44
47

Enterprise Data Warehousing With Sap BW - Overview

Uploaded by

Copyright:

Available Formats

You might also like

Enterprise Data Warehousing With Sap BW - Overview

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Enterprise Data Warehousing With Sap BW - Overview

Uploaded by

Copyright:

Available Formats

Enterprise Data Warehousing with SAP BW An Overview

White Paper Version 2.0 August 18th, 2003

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW

2003 SAP AG AND SAP AMERICA, INC.

BW Enterprise Data Warehousing Overview

This document is relevant for BW Versions 3.x.

This White Paper

2 Aims of the White Paper

2003 SAP AG AND SAP AMERICA, INC.

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW

Strategic Corporate BW Implementations

Architected BW Landscape Scenario

BWn Issues: Synergy Integration Consistency

BW Enterprise Data Warehouse Scenario

2003 SAP AG AND SAP AMERICA, INC.

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW

Why Do You Need A Corporate BW Strategy?

Redundancy and Data Mart (Solution) Focus

Information .... Requirements

SuccessfulData DataWarehousing Warehousingmeans means Controlled ControlledRedundancy Redundancy!! Successful

BW Application Project Team

Schema-Modeling InfoCubes/ ODS-Objects

Business Rules Transformation Extraction Sources

Illustration 2: Redundancy and Solution focus

2003 SAP AG AND SAP AMERICA, INC.

Redundancy and Multiple BW Landscape

Sub-Division A R/3 Global

Sub-Division B R/3 only Germany

Sales Europe R/3

Div Div Div .................... R/3 R/3 R/3

SuccessfulData DataWarehousing Warehousingmeans meansControlled ControlledRedundancy Redundancy !! Successful

Illustration 3: Redundancy and multiple BW landscape

2003 SAP AG AND SAP AMERICA, INC.

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW

4 Enterprise Data Warehousing with SAP BW

2003 SAP AG AND SAP AMERICA, INC.

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW

Aspects of a Data Warehouse Architecture

2003 SAP AG AND SAP AMERICA, INC.

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW

Non-SAP Source Persistent Staging Area SAP Source

Operational Data Store

Architected Data Marts

Archiving & Near-Line Storage

2003 SAP AG AND SAP AMERICA, INC.

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW

Operational Data Models and the Data Warehouse Data Model

A data warehouse consistency challenge

Not aligned operational data models

Consistent enterprise data model

Spoke BW Spoke BW Spoke BW

OutsideOutside-IN BW Landscape Global BW

2003 SAP AG AND SAP AMERICA, INC.

2003 SAP AG AND SAP AMERICA, INC.

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW

Data Store Architecture - BW Data Layer

SAP Business Information Warehouse

Architected Data Marts Extended Star Schemas