MIGRATION OF DATA

Transferring data from legacy applications or from old systems to new ERP application is an
important step for ERP implementation. This is something done as part of final preparation stage
of a project or during late realization. However, the planning for this need to start much earlier.
Success of an ERP project can depend a lot on quality of data that is going inside the system and
if proper care is not taken in loading the system with clean data, it can simply become “garbage
in and garbage out”. So, it’s worth spending time in data cleaning. There can be other
complications also during this stage like ERP systems may need data which is not existing at all
and this need to be created by the project team. For example, implementing planning modules
may need data on service level or safety stock requirement of an item which most of the
organizations may not maintain in their legacy application (this information is in the head of the
executives and may not be stored in any database) and these data need to be created? Similarly, a
company looking for an ERP implementation in a new plant may need to create new master data
on machines, cost centers, etc.

Data migration involves several challenges as shown in Figure 14.1 and discussed as follows:

 From which source systems data will come?

 How much data need to be migrated?
 How to clean the data?
 Who will verify and ensure that clean data is loaded into the system?
 Which electronic migration method or ETL tool to be used?
 What data can be transferred earlier and what need to be transferred just before going
live?
Classification of Data to be migrated

There are two sets of data which need to be migrated from legacy application to the new ERP
systems and these are Master Data and Transaction Data.

Master data is typically the data set which does not change every day like vendor, customer, and
materials, etc. It is important that only relevant master data is getting transferred, i.e. there is no
point in transferring the customers or suppliers from your old system with which you no longer
do business. In the similar manner, there is no point in transferring obsolete materials. These are
part of data cleaning activity and need to be taken seriously so that the ERP system is not getting
loaded with junk data.

Transaction data are the open items like stock, open purchase orders, contracts, sales orders, etc
that need to be transferred to the new ERP system. Again, the guiding principle here is to transfer
only relevant data, i.e. there is no point in transferring the purchase orders which already got
closed three months back, but those purchase orders against which vendors will give materials in
future need to be transferred to new system, otherwise goods cannot be received in the new
system against those open purchase orders.

Sometimes, it becomes important to do reporting, based on past data, i.e. you want to see how
the vendor had performed in the last five years. In those cases these old data from legacy
applications can be transferred to some data warehousing application and from there reporting
can be done. Old data from legacy applications should not be transferred to ERP application just
for reporting purpose as too much data makes system performance slow.

Examples of some master data objects which are taken up as part of data migration:

 Chart of accounts
 Cost centers
 Profit centers
 Asset master
 Tax master
 Material master—Finished, semi-finished, spares
 Price Masters
 Vendor master
 Customer master
 Employee master

Examples of some transaction data objects/open items which can be taken up as part of data
migration:

 Open internal orders

 Excise balances
  Material in transit
 Open service contracts and orders
 Open RFQs (request for quotations)
 Open contracts and purchase orders
 Open sales contracts and orders
 Loans and advances, contracts

Process to Migrate Data

Data can be migrated to the new ERP application system either manually or electronically. Both
the process of data migration is discussed as follows:

Manual Data Migration

Figure 14.2 shows basic process for manual data migration. For very small implementation, data
migration can be done manually as well where a set of people check the source system and then
manually punch data in new ERP application. This is time consuming, costly and having high
chance of error. For relatively large data size, manual migration is an impossible task.
Nowadays, in nine out of ten projects, this method is not followed.

Electronic Data Migration

This process is most popular these days as volume of data is not a con-strain here. This is faster,
cheaper and having lower chances of error. It is important to remember that even in case of
electronic migration, after the migration is complete; data need to be verified in the new system
manually. Manual verification is the last step even in case of electronic transfer. This is typically
a seven-step process as shown in Figure 14.3. The steps are as follows:

 Analysis
 Design
 Cleansing
 Extraction
 Transform
 Load
 Verification
Data Cleansing

Data cleansing is discussed here separately as this is a very important activity and often in ERP
projects, due to high time pressure, this activity is overlooked. There can be various problems
with the data, some of which are as follows:

 Same person with different spellings: Bannerjee, Bandopadhay, etc

 Multiple ways to denote the same company name: ABC Systems, ABC Pvt. Ltd,
ABCPL, etc.
 Same city with different names: Mumbai and Bombay, Calcutta and Kolkata
 Different account numbers generated by different applications for the same customer
 Required and mandatory fields left blank
 Product codes entered at store POS system are incorrect. In case of a problem, store
clerk had entered manually some dummy value like 9999999.
 Contradicting data

ETL Tools

Extraction, Transformation and Loading tools are widely used for data migration. These tools
can help in auto data cleaning based on rules, data extraction from multiple legacy applications
and finally data loading into ERP and can be useful for large ERP projects having huge data to
migrate. ETL tools can take out data from any source systems, i.e. legacy applications,
spreadsheets, standalone database, etc. Some of the leading vendors and products in this space
are:

 Business Objects (Product: Data Integrator)

  IBM (Product: Information Server)
 Informatica (Product: Power Centre)
 Oracle (Product: Oracle Warehouse Builder)
 Microsoft (Product: SQL Server Integration Services)

How ETL Tools Helps in Data Cleaning: ETL tools have advanced capabilities in data cleaning
like:

Capabilities to cleanse and standardize data: Capabilities like data standardization, cleansing,
and formatting helps in consistency in data capture and reporting.

Capabilities of data matching: Matching rule engines and algorithms helps in determining
whether different versions of similar data. For example, a customer record, location, or product
should be considered the same entity.

Capabilities of data merging and consolidation: Having determined that there is a positive
match, the next task is to delete the duplicate versions of the data and keep the one which is the
best version.

Capabilities to verify postal address: Data cleansing supports postal address verification and
enrichment (i.e. ensuring correct mailing address; street address, city, state, postal code, country,
etc.) by partnering with national postal services around the globe. Some vendors specialize in
specific countries or regions.

Capabilities to verify phone/email active use: Vendors provide verification capabilities of

customer phone numbers and email addresses.