SAP Data Migration Hits and Misses by

Feroze Arif
Data migration guru, John Morris, wrote data migration is a business issue and not a
technical issue. Having been through many data migration projects myself, couldn!t agree
more. "hen choosing a framewor# for data migration one has to #eep this in mind.
$urthermore when it comes to migrating data into %&', one needs to ta#e into consideration
a few other truisms, such as(
).Data *uality is not achieved all at once but over repeated iterations
+."hile data migration itself is a one,time event, within the implementation repeated cycles
of e-traction, transformation and loading ./012 into multiple %&' environments are re*uired
3.1oading into %&' is a time consuming process
4ac# in the day data migration was considered an unglamorous job that paid little.
mplementation consultants preferred to fight more glorious battles such as business
process re,engineering. t was left to lowly cler#s and programmers to du#e it out with data.
t too# a while, but system integrators finally reali5ed that a fabulous implementation with
great processes could be *uic#ly brought to its #nees by poor data. 0his reali5ation has
spar#ed the creation of numerous approaches on how to migrate data. 6ne approach that
has gained popularity and has done so rightfully is the %&' 4est 'ractices for Data
Migration.
0he %&' 4est 'ractices for Data Migration framewor# is based on the %&' Data %ervices
application which is their /01 tool. 0he main components of the framewor# are %&' Data
%ervices, a staging database, a set of migration templates, a set of corresponding mapping
templates, a tool to map source values to target values and a methodology to be followed.
0he migration templates are 7programs! or in %&' Data %ervices parlance 7jobs! that %&'
Data %ervices e-ecutes. 0hese jobs fall into multiple categories. 6ne job category
downloads %&'!s configuration tables into the staging area. 0he framewor# uses the
downloaded configuration 8 loo#up data to validate the data to be migrated. & second job
category handles /01 of data into SAP. 0as#s such as reconciliation of data loads, status
chec#s and creation of staging area data stores are handled by other job categories. 0he
framewor# also supports creation of %&' 4usiness6bjects9 :niverses used for reporting on
the progress of the data migration project.
Page 1 of 3
0he jobs have a uniform and modular design that allows them to support the data migration
process. 0he standardi5ed design ma#es maintenance of the jobs easier and significantly
reduces the duration of the inevitable #nowledge transfers between developers that occur
on a typical data migration project.
$rom a developer!s standpoint data migration programs can be thought of as e-ecuting a
set of steps in a specific se*uence. 0he typical steps are field mapping, value mapping or
transformation, validation and finally loading. /ach of these %&' 4est 'ractices for Data
Migration data service jobs comprises of parts that mirror the above tas#s and se*uence. n
the first part the source fields are mapped to the target fields. n the second part the data is
validated. 0he third part transforms source values into target values and the final part loads
the data into %&' using D6;s.
1i#e other framewor#s, %&' 4est 'ractices for Data Migration jobs validate data. 0he
differentiator is how the validation is implemented when using %&' 4est 'ractices for Data
Migration. 0hree categories of validation are applied to the data. 0he data is validated to
chec# if mandatory fields are populated< the format and the values of data are acceptable to
%&'. "hile these seem li#e standard validations, the framewor# offers two advantages.
$irst, it is not necessary to load the data into %&' to determine the validity of the data. %&'
4est 'ractices for Data Migration allows data to be validated within its staging area. &nyone
who has loaded data into %&' and has spent hours waiting for data to load will appreciate
the fact that the %&' 4est 'ractices for Data Migration framewor# allows you to validate the
data as many times as re*uired at the clic# of a button in a very short period of time.
%econd, %&' 4est 'ractices for Data Migration allows you to identify all issues with an
individual data record at one time.
&fter running the validation, the framewor# stores invalid data in a separate table. /ach
record is tagged with the reason for validation,failure in plain /nglish. 0hese records can be
provided to the business users who can then fi- the error. 0he corrected records can then be
passed through the jobs again and if any data issues are encountered again the data can be
returned to the business for fi-ing. 0his process can be repeated any as often as re*uired
until all the data *uality issues are resolved. SAP Best Practices for Data Migration supports
close and frequent interaction with the business. 0his support of the interaction shows how
the framewor# incorporates the aphorism that data migration is a business issue and not a
technical issue.
"hile validation is possible, there is a short coming as well. ;ross field validation is not
available. 0his has to be custom built. My colleague ;huc# %chardong recommends that
these custom e-tensions be built outside of the %&' 4est 'ractices for Data Migration
templates. 0hough not very difficult to build, it would!ve been nice if the framewor# too#
into account such validations. &nother drawbac# of the framewor# is that it does not provide
an interface that allows the business users to correct the data. &t the moment the data can
be provided in the lingua franca of all business users e.g. Microsoft /-cel=. However,
handling numerous files and #eeping trac# of which files are current is a headache that any
data migration practitioner could do without.
%&' implementations span multiple months if not years and involve multiple environments
and test cycles. Hence a good framewor# that supports a %&' data migration should be
capable of repeating the data migration tas#s often and in multiple environments, an area
which many framewor#s miss. >ot so with the %&' 4est 'ractices for Data Migration. 6nce
the %&' Data %ervice jobs are customi5ed and tested, the jobs can be e-ecuted any number
of times. &nyone who has wor#ed in an %&' implementation #nows that there can be
different configurations in different %&' 7;lients!. 1ast minute configuration changes are the
Page 2 of 3
rule rather than the e-ception. "hen it comes to multiple environments and changes, the
framewor# downloads the %&' configuration information each time and uses it to validate
the data before loading into that environment. 0his ensures that changes to the %&'
configuration are always ta#en into account and the data that is provided for loading, will
load into the %&' system.
6ne of the major challenges of a data migration is loading data into %&'. 0he 1egacy
%ystem Migration "or#bench .1%M"2 is the preferred option for loading data into %&'.
However, the process is lengthy and errors thrown out by the 1%M" tend to be cryptic.
0here are also instances when a record is being loaded and 1%M" will stop loading the
record at the first error it encounters. f there are additional errors, they will be pic#ed up
only when the data is loaded again into %&'. %ince loading data is itself a time consuming
process, finding all the errors could turn into a long and painful process. 0his is where the
framewor# really shines and proves its value. ?alidation within the staging area and
validation of all fields in a record at one time eliminates the issue of having to repeatedly
load the data into %&' to discover all the errors. %&' Data %ervices in an %&' environment
provides a variety of ways to load data into the system. 0he framewor# supports D6;s and
in cases where traditional 1%M" methods are re*uired, the framewor# can also provide
validated te-t files that can be loaded into %&'. D6;s are a fast and efficient way of loading
data into %&'. 4y just changing a few parameters in the %&' Data %ervices configuration,
the environment into which the framewor# loads data can be changed. 0his literally allows
users to load data into %&' at the touch of a button.
n addition to the above there are other advantages of using the framewor#. 0he framewor#
includes pre,built templates for various %&' data objects. 0his reduces the development
cycle. n my e-perience !ve seen a reduction of about 3@A in terms of the effort re*uired
to build /01 jobs for data migration. &lso these pre,built templates contain most of the
validations that are re*uired for %&', which reduces the probability of missing a particular
validation and provides peace of mind to developers. 6f course the ease of use of the
templates comes at a price. %ince they necessarily have to cater to multiple re*uirements,
they are built with a one si5e fits all mentality. 0he templates contain all the fields that are
available for a %&' object. t becomes tedious when the input contains only a few fields.
Bemoving the e-tra fields from the template or setting them to null can be a long drawn out
process. 0his perhaps is the reason why !ve seen a 3@A decrease in effort and not a
greater effort savings.
n summary, the %&' 4est 'ractices for Data Migration is a solid and reliable framewor# that
can be and has often been used to migrate data successfully. %&' 4est 'ractices for Data
Migration has a superior validation method within the staging area. 0he integration with %&'
allows close interaction with business users even as it supports technical tas#s such as
repeated loading of data into multiple %&' environments. &s a practitioner feel this is one
of the better framewor#s around.
&s an %&' partner and data e-perts, :topia has e-tensive #nowledge of data objects and
how they align to %&' implementations for specific %&' modules learn more about data
migration jumpstart services.
Page 3 of 3