Overview of Data Conversions

Author Date

: Abhijit Kadu : 17-MAY-2005

1 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

Objectives

Define & describe data conversions Types of data conversions Conversion strategy Object development strategy Coding standards Error handling mechanism Tips for performance tuning General guidelines Summary

2 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

Define & Describe Conversions

Conversion

LEGACY

ORACLE APPLICATIONS 11i

Converting the existing data from legacy into new system. Applying set of data / processing rules Select,Derive,Default,Required Ensuring business continuity flawlessly from day one

3 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

Types of Conversions
Manual Conversions Automated Conversions Data Loader Programmatic Standard Non-Standard Direct data load

4 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

Conversion Strategy
Conversion objects finalization
Identify dedicated oracle / legacy tech & func team, business owners Strategy for each object followed by code design. Data extractions & program development strategy at legacy end Pre-requisite application setups by consultants. Determining sequence of running conversion objects including Manual conversion. Validation of extracted data from legacy, Programmatically and also manually by functional users. Running conversion programs multiple times with progressive data sets. Reconciliation of converted data statistically using SQL scripts User acceptance of converted data. Provide data fix scripts after major milestone instead of code change

5 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

Object development strategy

CV40 Checklist - User ten points checklist to approve conversion
functional specification. Insists for legacy sample records as an CV040 attachment Prepare object execution steps Instruction about handling object Ad hoc changes Preprocessing utilities to handle preprocessing activities API Common package for common procedures. Generic & standardized technical design Standardized generic coding framework / approach Discuss conversion staging table structure with legacy team to avoid Integration issues.

6 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

Coding Standards
Standardized generic coding framework / approach
Avoid statements like for update of and current of into cursor. Keep code as smaller, compact, simple as possible Avoid re-building of validation that are handled by open Interfaces. Use custom API and reduce coding time. Create reconciliation scripts along with code for analyzing conversion data statistics. Avoid hard coding of multiple permutations and combination of conditions in code, Instead use database table encapsulated by function to define such rules. This gives more Flexibility to change rule on the fly without changing the code. Provide Validate / Populate steps to the programs. This gives better control at every steps. Avoid frequent code changes after major milestones Register all conversion concurrent under one request group & assigned to one responsibility. Code should be peer reviewed by Technical lead.
7 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

Error handling mechanism

Staging table should contain following error tracking columns
Record ID Process flag Concatenated error messages Request ID Who Columns Create common error table should contain following columns Conversion Code / Object name Record ID & Request ID Table & Column name on which error occurred Value, which is cause of error Process flag & Error Message Who Columns Create error logging API procedure Create common error report and concurrent program Error table purge utility
8 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

Tips for performance tuning

Rollback segment Sizing according to the data volume. Perform intermediate commits at regular interval. Avoid expensive statements within cursor Loop. Use of Ids instead of code, making proper joins Dont use ORDER BY or DISTINCT clause while selecting records in cursor for processing. Unless there is specific requirement. Capture fetched Ids once into variable for future use, instead of fetching them again and again. Any custom program taking more then 2 hour of processing time will be candidate for tuning. Gather schema , table, index level statistic at right moment of program execution. Develop utility tool to collect statistic of objects listed in underlying table. Build proper indexes on staging or base tables. Avoid unnecessary, too many indexes If required tune your program using trace file & explain plan
9 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

General guidelines
Avoid programmatic conversion of lowvolume data Prefer to process data using standard interfaces/API to ensure data integrity. Upfront highlighted the risk of conversion involving base table update and get customer buy in as well. Leverage custom Interface code (new or existing) for conversion. Get customer buy-in for not converting historical data. As a result, data only relevant during cut-over should be part of conversion. Insist legacy to provide clean data keeping hands off from getting involved in data extraction. Use common piece of codes,API. Quality standards - CV40 Checklist, Coding standard, Conversion responsibility, User, Application etc were published.

10 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

10

General guidelines
Insist to validated extracted data, In staging table itself, by Functional users to avoid fixing issues latter. Maximum effort was given to reconcile converted data in between conversion execution at predefined stages to be sure Things are OK up-to this stage Perform multiple rounds of test Fire (TF1, TF2, SIT, TF3, Dry Run/TF4) and run conversion programs with progressive data sets from production snapshots. This ensured that previous runs error fixes. All these fixes were recorded into Roadmap documents After each run perform functional data validation as mandatory step.

11 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

11

Summary

Define & Describe Data Conversions Types of Data Conversions Conversion Strategy Object development Strategy Coding standards Error handling mechanism Performance Tuning General guidelines

12 4/1/2013 GECIS IT Services Proprietary and Confidential

Classification: Gecis Internal

12