GTCC Data Migration

Automation / Initiatives
15-Oct-2020
GTCC Data Migration Team

Public
Imagination at work
1. System Availability - SAHI Automation
2. Data Validation - Validax
3. Data Availability - Handbook
4. Pre- Requites - Dashboard

1. System Availability

Problem Statement:
• System / Integrations unavailable at Time
• Integrations/ Items that are managed by External Teams, when not working
becomes a hurdle for the team / Testing team and adds cycle time

Title or Job Number | XX Month 201X 2

 - SAHI Automation
• Unless tested and Observed – sometimes it’s too late to coordinate / React •
Common Failure Modes – 3DSpace, TCT Tag, Convert Logic, Indexing, FCS is down

The Idea :
• A Timely Test before IST/EDT work hours for first-hand info if system is Unavailable
• This Info Available to everyone in the team so anyone can take action
• No Manual Effort for this Activity

Title or Job Number | XX Month 201X 3

 - SAHI Automation
1. System Availability
• Basic Tests -
– Login →Nodes are up and Available
→License server is up & TCT Integration is
– Create an Object & ECCN Tag
Up
– File Check-in & Convert Logic →FCS is Up
– Search the Object →Partial Indexing Job is up and Working
• SAHI Automation Scripts Hosted in Sandbox
• Sends Success/ Failure Info to the entire Team
• Scheduled to run every 6 Hours covering EST/IST start/end of work

Title or Job Number | XX Month 201X 4

 - SAHI Automation

1. System Availability
Success Message :

Title or Job Number | XX Month 201X 5

Failure Message :

Benefits :

• No Manual Efforts, Automated Jobs

• Quicker Turnaround in case of Issues
• Maximum System Availability

Title or Job Number | XX Month 201X 6

 - SAHI Automation
2. Data Validation - Validax

Problem Statement:
• Manual Execution of Queries – needs dedicated time and effort from a Developer
• When Manual – Chances of Missing out Validations.
• Regression is not always done – due to time limitations
• Less Time for validation due to the Milestones / other unplanned delays

The Idea :
• One Time Development – Use Every time
• Less or No Manual / Human Effort
• Validate each Phase of ETL in DM Process.

Title or Job Number | XX Month 201X 7

• Regression Testing - All Defects Validation at Table level before Data Load

Title or Job Number | XX Month 201X 8

2. Data Validation - Validax
The migration process of GTCC is classified in 4 sections.

• LX TABLE & ODS Data to PLMWHT_STAGING schema

• PLMWHT_STAGING to GTCC_PLM_STAGING schema
• Staging Environment to QA2 or DEV2 environment.

• Data Loaded outside of Adaplet (TCL / MQLs)

Checkpoints:
1. During Data Profiling when a new energy dump is provided
The Data Migration team creates Flat or DM tables from the LX or MX Tables. This will validate
the Data Count and attributes between LX tables to DM Tables.
2: Data Validations after the Tran Tables are Loaded
Once the Target tables are loaded, This will validate all the object and Relationship basic validations.
3: Defect tracking and validation through Validax
 Once a defect is raised in Rally one query will be added to validate the same in Validax. It will be
run before each run so even if the defect is closed in rally we will continue to validate them

2. Data Validation - Validax

✓Examples of ETL Level Data Validations

✓ LXTYPE Verification

✓ Count Comparison from LXBO to Staging

✓ Special Character Check

✓ Count Comparison for Source data going as is

✓ Distinct ID Comparison from Staging to Target

✓ Duplicate TNR Verification in staging and Target

✓ Transformed Type verification with Staging

✓ State and Policy range in staging and Target

✓ Data Comparison for attributes getting migrated as is

✓ Special Character Transformation in Target

✓ Date Format Transformation in Target

✓ Next Revision missing Verification in Target

 -
✓ Count Verification

✓ Duplicate LXOID Validation

2. Data Validation Validax

Title or Job Number | XX Month 201X 11

Demo On DM Sandbox
 -
3. Data Availability Handbook

Problem Statement:
• Testing Team has dependency on Development Team
• Wait Time / Multiple Iterations if any new data is needed
• Multiple Scenarios – need multiple Data Sets

The Idea :
• On-Demand / Query Builder Tool

Benefits :

Title or Job Number | XX Month 201X 13

 -
• Zero Wait Time for Test Data Availability

3. Data Availability Handbook

Title or Job Number | XX Month 201X 14

 -
Portal Link : http://3.29.200.185/Default.asp
* Restricted to DM Team Only

4. Data Pre- Requites Dashboard

Problem Statement:

• Several Parallel Tracks of Data Clean Up discussion/ Tracking

• Lack of one common place with Statistics

The Idea :

• One Common Dashboard for all the Pre-Requisite DM Data Clean Ups
• Weekly Availability of Statistics for Functional Team and/or TPMs to take Action
• Traction on Action Items is intact
• Automateg generation of Statistics in a Dashboard – No additional Human effort
Sl No Process Area Test Case Data Count Generation Date
1 Generic Objects with Multiple Latest Revision ID 47,231 14-Oct-20

2 Parts & Bom Parts without Part Master 893 14-Oct-20

3 Parts & Bom Parts without Part Family 41 14-Oct-20

4 Parts & Bom Parts Connected to Multiple MEPs 4603 14-Oct-20

Title or Job Number | XX Month 201X 15

 -
5 Parts & Bom Part with numeric revision other than 3 digits 2728 14-Oct-20

6 Parts & Bom Parts Type,Policy Mismatch in PG 96 14-Oct-20

7 Document Objects with Same Name in Multiple Document Types 152290 14-Oct-20
8 XCAD UG Model with Same Name as Acad Drawing (Tech Doc) 50 14-Oct-20
Work In Progress. The Target is to make it available by End of October

Title or Job Number | XX Month 201X 16

 Thank You

Contact : @POWER GTCC Data Migration Team

Title or Job Number | XX Month 201X 17