Professional Documents
Culture Documents
PWX 85 Interfaces For Power Center
PWX 85 Interfaces For Power Center
Informatica® PowerExchange®
(Version 8.5)
PWX_Interfaces_for_PowerCenter.book Page ii Wednesday, October 3, 2007 9:53 AM
This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing
restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be
reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. This
Software is protected by U.S. and/or international Patents and other Patents Pending.
Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as
provided in DFARS 227.7202-1(a) and 227.7702-3(a) (1995), DFARS 252.227-7013(c)(1)(ii) (OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR
52.227-14 (ALT III), as applicable.
The information in this product or documentation is subject to change without notice. If you find any problems in this product or documentation, please report
them to us in writing.
Informatica, PowerCenter, PowerCenterRT, PowerCenter Connect, PowerCenter Data Analyzer, PowerExchange, PowerMart, Metadata Manager, Informatica
Data Quality, Informatica Data Explorer, Informatica Complex Data Exchange and Informatica On Demand Data Replicator are trademarks or registered
trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade
names or trademarks of their respective owners.
DISCLAIMER: Informatica Corporation provides this documentation “as is” without warranty of any kind, either express or implied, including, but not limited
to, the implied warranties of non-infringement, merchantability, or use for a particular purpose. Informatica Corporation does not warrant that this software or
documentation is error free. The information provided in this software or documentation may include technical inaccuracies or typographical errors. The
information in this software and documentation is subject to change at any time without notice.
Table of Contents
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
About this Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Document Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii
Other Informatica Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Visiting Informatica Customer Portal . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Visiting the Informatica Web Site . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Visiting the Informatica Knowledge Base . . . . . . . . . . . . . . . . . . . . . . xiii
Obtaining Customer Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
iv Table of Contents
PWX_Interfaces_for_PowerCenter.book Page v Wednesday, October 3, 2007 9:53 AM
Configuring Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
DB2 Batch Mode Relational Database Connections . . . . . . . . . . . . . . . . . . . 84
DB2 CDC Mode Application Connections . . . . . . . . . . . . . . . . . . . . . . . . . 87
NRDB Batch Mode Application Connections . . . . . . . . . . . . . . . . . . . . . . . 90
NRDB CDC Mode Application Connections . . . . . . . . . . . . . . . . . . . . . . . 91
NRDB Lookup Relational Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
MSSQL Batch Mode Relational Connections . . . . . . . . . . . . . . . . . . . . . . . 95
MSSQL CDC Mode Application Connections . . . . . . . . . . . . . . . . . . . . . . 97
Oracle Batch Mode Relational Connections . . . . . . . . . . . . . . . . . . . . . . . 100
Oracle CDC Mode Application Connections . . . . . . . . . . . . . . . . . . . . . . 102
Sybase Batch Mode Relational Connections . . . . . . . . . . . . . . . . . . . . . . . 105
Configuring Connections Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Common Connection Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Batch Application and Relational Connection Attributes . . . . . . . . . . . 108
CDC-Specific Connection Attributes . . . . . . . . . . . . . . . . . . . . . . . . . 110
Understanding Commit Processing with PWXPC . . . . . . . . . . . . . . . . . . . 121
Table of Contents v
PWX_Interfaces_for_PowerCenter.book Page vi Wednesday, October 3, 2007 9:53 AM
vi Table of Contents
PWX_Interfaces_for_PowerCenter.book Page vii Wednesday, October 3, 2007 9:53 AM
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
List of Figures
Figure 1-1. PWXPC Batch Mode Extraction Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . .. . .. 5
Figure 1-2. PWXPC Change Mode Extraction Data Flow . . . . . . . . . . . . . . . . . . . . . . . .. . .. 6
Figure 1-3. PWXPC Real Time Mode Extraction Data Flow . . . . . . . . . . . . . . . . . . . . . .. . .. 7
Figure 3-1. Import from PowerExchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 19
Figure 3-2. Import from PowerExchange - Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 20
Figure 3-3. Import from PowerExchange - DB2/390 source . . . . . . . . . . . . . . . . . . . . . . .. . . 22
Figure 3-4. Import from PowerExchange - DB2/400 or DB2/UDB source . . . . . . . . . . . .. . . 23
Figure 3-5. Import from PowerExchange - DB2390 Select Datamaps List . . . . . . . . . . . . .. . . 25
Figure 3-6. Import from PowerExchange - Microsoft SQL Server source . . . . . . . . . . . . . .. . . 27
Figure 3-7. Import from PowerExchange - MSSQL Select Datamaps List . . . . . . . . . . . . .. . . 29
Figure 3-8. Import from PowerExchange - Oracle source . . . . . . . . . . . . . . . . . . . . . . . . .. . . 31
Figure 3-9. Import from PowerExchange - Oracle Select Datamaps List . . . . . . . . . . . . . .. . . 33
Figure 3-10. Import from PowerExchange - Sybase source . . . . . . . . . . . . . . . . . . . . . . . .. . . 35
Figure 3-11. Import from PowerExchange - Sybase Select Datamaps List . . . . . . . . . . . . .. . . 37
Figure 3-12. Non-Relational Source Definition with Multiple Records . . . . . . . . . . . . . . .. . . 39
Figure 3-13. Import from PowerExchange - Non-Relational Source . . . . . . . . . . . . . . . . .. . . 41
Figure 3-14. Import from PowerExchange - Single Record Source Data Maps . . . . . . . . . .. . . 43
Figure 3-15. Import from PowerExchange - Multiple Record Source Data Maps . . . . . . . .. . . 44
Figure 3-16. Import from PowerExchange - Non-Relational Target . . . . . . . . . . . . . . . . .. . . 45
Figure 3-17. Import from PowerExchange - Non-Relational Target Select Datamaps List .. . . 47
Figure 3-18. Non-Relational Table - Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 48
Figure 3-19. Non-Relational Table - Metadata Extensions . . . . . . . . . . . . . . . . . . . . . . . .. . . 49
Figure 3-20. Import from PowerExchange - CDC Datamaps . . . . . . . . . . . . . . . . . . . . . .. . . 54
Figure 3-21. Import from PowerExchange - CDC Select Datamaps List . . . . . . . . . . . . . .. . . 55
Figure 3-22. Extraction Map Table - Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 57
Figure 3-23. Extraction Map Table - Metadata Extensions . . . . . . . . . . . . . . . . . . . . . . . .. . . 58
Figure 3-24. Multi-record Non-Relational Source Definition . . . . . . . . . . . . . . . . . . . . .. . . 69
Figure 3-25. Group Source Mapping Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 70
Figure 4-1. Relational Connection Browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 81
Figure 4-2. Application Connection Browser . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 82
Figure 4-3. Application Connection Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 83
Figure 5-1. Session Mapping Tab - Batch VSAM Reader . . . . . . . . . . . . . . . . . . . . . . . . .. . 128
Figure 5-2. Session Mapping Tab - DB2 Readers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 131
Figure 5-3. Session Properties Tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 135
Figure 5-4. Session Mapping Tab - Extraction Map Source . . . . . . . . . . . . . . . . . . . . . . .. . 136
Figure 5-5. Session Mapping Tab - Relational Source . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 138
Figure 5-6. Session Mapping Tab - Relational Targets . . . . . . . . . . . . . . . . . . . . . . . . . . .. . 141
Figure 5-7. Session Mapping Tab - Non-Relational Targets . . . . . . . . . . . . . . . . . . . . . . .. . 143
Figure 6-1. Application Connection - Number of Restart Token Files . . . . . . . . . . . . . . .. . 173
Figure 7-1. Primary key updates from a source relational table . . . . . . . . . . . . . . . . . . . . .. . 176
List of Figures ix
PWX_Interfaces_for_PowerCenter.book Page x Wednesday, October 3, 2007 9:53 AM
x List of Figures
PWX_Interfaces_for_PowerCenter.book Page xi Wednesday, October 3, 2007 9:53 AM
List of Tables
Table 1-1. Functional Comparison between PWXPC and PowerExchange ODBC . . . . . . . . . . . 2
Table 1-2. PowerExchange Client for PowerCenter Extract and Load Capabilities . . . . . . . . . . . 3
Table 1-3. PowerExchange Database Types Supporting Group Source . . . . . . . . . . . . . . . . . . . 4
Table 1-4. PowerExchange ODBC Extract and Load Capabilities . . . . . . . . . . . . . . . . . . . . . . . 8
Table 2-1. PowerExchange and PWXPC Coexistence Matrix . . . . . . . . . . . . . . . . . . . . . . . . . 16
Table 3-1. Attributes of Fields in a Non-Relational Source Definition . . . . . . . . . . . . . . . . . . 48
Table 3-2. Non-Relational Source Definition Metadata Extensions . . . . . . . . . . . . . . . . . . . . 50
Table 3-3. Attributes of Fields in a Extraction Map Definition . . . . . . . . . . . . . . . . . . . . . . . 57
Table 3-4. Extraction Map Definition Metadata Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Table 4-1. Connection Types for Extracting Source Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Table 4-2. Connection Types for Loading Target Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Table 4-3. PWX DB2390, DB2400, and DB2UDB Relational Database Connection Attributes 84
Table 4-4. DB2390, DB2400, and DB2UDB CDC Mode Application Connection Attributes . 87
Table 4-5. NRDB Batch Mode Application Connection Attributes . . . . . . . . . . . . . . . . . . . . 90
Table 4-6. NRDB CDC Mode Application Connection Attributes . . . . . . . . . . . . . . . . . . . . . 91
Table 4-7. NRDB Lookup Relational Connection Attributes . . . . . . . . . . . . . . . . . . . . . . . . . 94
Table 4-8. MSSQL Batch Mode Relational Connection Attributes . . . . . . . . . . . . . . . . . . . . . 95
Table 4-9. MSSQL CDC Mode Application Connection Attributes . . . . . . . . . . . . . . . . . . . . 97
Table 4-10. Oracle Batch Mode Relational Connection Attributes . . . . . . . . . . . . . . . . . . . . 100
Table 4-11. Oracle CDC Mode Application Connection Attributes . . . . . . . . . . . . . . . . . . . 102
Table 4-12. Sybase Batch Mode Relational Connection Attributes . . . . . . . . . . . . . . . . . . . . 105
Table 4-13. Encryption and Compression Connection Attributes . . . . . . . . . . . . . . . . . . . . . 107
Table 4-14. Pacing Size Connection Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Table 4-15. Convert Character Data Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . . . 108
Table 4-16. Write Mode Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Table 4-17. Retrieve PWX Log Entries Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . 110
Table 4-18. Image Type Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Table 4-19. Event Table Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Table 4-20. CAPI Connection Override Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . 113
Table 4-21. Idle Time Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Table 4-22. Change and Real-time Mode Restart Connection Attributes . . . . . . . . . . . . . . . 115
Table 4-23. UOW Count Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Table 4-24. Real-Time Flush Latency Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . . 117
Table 4-25. Commit Threshold Connection Attribute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Table 5-1. Session Properties for Non-Relational Targets . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Table 6-1. Default Starting Extraction Points for Sources . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Table 6-2. Recovery Table SQL Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Table 6-3. CDC Sessions - Recommended Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
Table 10-1. Connection Types for Extracting Source Data . . . . . . . . . . . . . . . . . . . . . . . . . . 203
Table 10-2. Target Database Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
List of Tables xi
PWX_Interfaces_for_PowerCenter.book Page xii Wednesday, October 3, 2007 9:53 AM
Preface
PowerExchange Batch provides the extract and apply functionality required to support data
warehouse and migration initiatives. PowerExchange has become the standard in this area,
demonstrating excellent performance, scalability, productivity and ease-of-use.
PowerExchange Change provides the ability to capture changes from transactional databases
and make them available at user-defined intervals. By capturing changes in source data, you
can now update your data warehouse on a more regular basis without overloading your IT
infrastructure. PowerExchange Change supports capture of changed data on a wide variety of
platforms and databases and does not require a separate set of database tables as a staging area.
Changes are available at separate intervals, to different targets, greatly enhancing flexibility
and performance. Changes can be presented in condensed form, providing just the net change
over the specified interval.
PowerExchange Real-time provides real-time access to changes streamed from transactional
databases. PowerExchange Real-time requires no additional programming to complete the
integration picture.
Change data captured by PowerExchange Change and PowerExchange Real-time is available
to PowerCenter. Changes can be sourced in relational format and can be distributed to
message queues or applied to targets through PowerCenter.
xi
PWX_Interfaces_for_PowerCenter.book Page xii Wednesday, October 3, 2007 9:53 AM
Document Conventions
This guide uses the following formatting conventions:
italicized monospaced text This is the variable name for a value you enter as part of an
operating system command. This is generic text that should be
replaced with user-supplied values.
Warning: The following paragraph notes situations where you can overwrite
or corrupt data, unless you follow the specified procedure.
bold monospaced text This is an operating system command you enter from a prompt to
run a task.
xii Preface
PWX_Interfaces_for_PowerCenter.book Page xiii Wednesday, October 3, 2007 9:53 AM
Preface xiii
PWX_Interfaces_for_PowerCenter.book Page xiv Wednesday, October 3, 2007 9:53 AM
Use the following telephone numbers to contact Informatica Global Customer Support:
North America / South America Europe / Middle East / Africa Asia / Australia
xiv Preface
PWX_Interfaces_for_PowerCenter.book Page 1 Wednesday, October 3, 2007 9:53 AM
Chapter 1
1
PWX_Interfaces_for_PowerCenter.book Page 2 Wednesday, October 3, 2007 9:53 AM
Overview
You can use the following interfaces to extract and load data through PowerExchange when
using PowerCenter:
♦ PowerExchange Client for PowerCenter (PWXPC). Chapters 2-6 describe the
PowerExchange Client for PowerCenter. PowerExchange Client for PowerCenter is fully
integrated into PowerCenter as a native plug-in. You can use it to extract and load data
through PowerExchange for a variety of data types on a variety of platforms. For a
complete list, see Table 1-2 on page 3.
♦ PowerExchange ODBC. Chapters 8-12 describe the PowerExchange ODBC interface.
You can use PowerExchange ODBC connections with PowerCenter to extract and load
data through PowerExchange for a variety of data types on a variety of platforms. For a
complete list, see Table 1-4 on page 8.
For an overview of PowerExchange, see the PowerExchange Getting Started manual.
For an overview of PowerCenter, see the PowerCenter Getting Started manual.
PWXPC provides additional functionality that is not available with ODBC. The following
table compares the interface functionality of the PowerExchange Client for PowerCenter and
PowerExchange ODBC:
Yes No Save target data and CDC restart CDC restart information is stored in the same
information in a single commit database as the relational target table in a
single commit providing guaranteed CDC restart
and recovery.
Yes No PowerCenter graceful stop support for real PowerCenter stops real time sessions after all
time sessions data in the pipeline is written to the targets.
Yes No Automatically process change indicator Each changed data record indicates whether it
is an insert, update, or deletes. You must use
an Update Strategy transformation to process
inserts, updates, and deletes if the change
indictor is not used automatically.
Yes No Modify the PowerExchange file name in the PowerExchange normally uses the file name
source definition from the PowerExchange data map. Changing
the file name in the PowerCenter source
definition overrides the PowerExchange data
map.
Table 1-2. PowerExchange Client for PowerCenter Extract and Load Capabilities
PWXPC uses PowerExchange Group Source functionality to extract changed data from the
change stream. PWXPC also uses Group Source to extract data from VSAM data sets and
sequential files containing multiple record types. Group Source reads all data from the same
physical source in a single pass. PWXPC connections provides enhanced performance over
ODBC connections by using Group Source to eliminate multiple passes of the source data. As
a result, PWXPC connections also reduce PowerExchange resource consumption on the
source or extraction platform. For detailed information about Group Source, see
“PowerExchange Group Source” on page 68.
Note: Some PowerExchange sources do not support Group Source.
Table 1-3 indicates which PowerExchange database types support Group Source:
Database Type Batch Mode CDC Change Mode CDC Real Time Mode
The PowerExchange Client for PowerCenter extracts relational and non-relational data in the
following modes:
♦ Batch
♦ Change Data Capture (CDC) - Change
♦ Change Data Capture (CDC) - Real Time
Batch Mode
Use PWX Batch application and relational connections to extract and load data for relational
databases and non-relational data sets and files through PowerExchange. PWXPC connects to
PowerExchange using the PowerExchange Call Level Interface (SCLI).
You can extract all records in multiple record type VSAM and sequential (flat) data sets with
a single pass of the data. PWXPC uses PowerExchange Group Source functionality to do this.
In contrast, ODBC connections read a single record type at a time requiring multiple passes
of the data.
The following diagram shows the data flow of source data from PowerExchange through
PWXPC and PowerCenter to the target tables:
reconstructs the units of work (UOW) into complete and consecutive UOWs.
PowerExchange Condense then writes these UOWs to condense files.
PWXPC uses the PowerExchange Capture Extraction (CAPX) Access Method to extract
changes from the condense files when you use PWX CDC Change connections. PWXPC uses
the PowerExchange Capture Extraction Real-Time (CAPXRT) Access Method to extract
changes when you use PWX CDC Real Time connections. The following diagram shows the
data flow of condensed changed data from PowerExchange through PWXPC and
PowerCenter to the target tables:
the units of work (UOW) into complete and consecutive UOWs. The UOW Cleanser is
incorporated into PowerExchange UDB and SQL Server CDC.
PWXPC uses the PowerExchange Capture Extraction Real-Time (CAPXRT) Access Method
when you use PWX CDC Real Time connections. The following diagram shows the data flow
of source changed data from PowerExchange through PWXPC and PowerCenter to the target
tables:
PowerExchange ODBC
PowerExchange provides a thin ODBC driver that you can use with PowerCenter. The
Integration Service connects to PowerExchange locally or remotely to extract or load data
through the PowerExchange ODBC interface. You can extract and load relational and non-
relational data. You can also extract changed data. The PowerExchange ODBC interface does
not support Group Source.
The PowerExchange ODBC interface extracts relational and non-relational data in the
following modes:
♦ Batch
The Integration Service extracts and loads data from a relational table or non-relational file
through PowerExchange. The PowerExchange ODBC interface reads a single type in
multiple record type VSAM and sequential files requiring multiple passes of the data to
read all record types.
♦ Change Data Capture (CDC) - Change
The Integration Service extracts changed data from condense files through
PowerExchange. The PowerExchange ODBC interface reads all of the changes captured in
condense files since the last extraction session. The extraction session ends after all of the
captured changes are read. PowerExchange maintains restart information in the change
information (CDEP) file on the source PowerExchange platform. The PowerExchange
ODBC interface has limited restart capability.
♦ Change Data Capture - Real Time
The Integration Service extracts changed data in real time from the change stream. You
run real time extractions run for a specified time period. PowerExchange maintains restart
information in the change information (CDEP) file on the source PowerExchange
platform. The PowerExchange ODBC interface has limited restart capability.
Table 1-4 shows the PowerExchange ODBC extract and load capabilities:
PowerExchange ODBC 9
PWX_Interfaces_for_PowerCenter.book Page 10 Wednesday, October 3, 2007 9:53 AM
Chapter 2
PWXPC - Installation
11
PWX_Interfaces_for_PowerCenter.book Page 12 Wednesday, October 3, 2007 9:53 AM
Overview
This chapter describes how to install and configure PowerExchange Client for PowerCenter
(PWXPC).
Installation Steps
The PowerExchange Client for PowerCenter (PWXPC) is installed when you install
PowerCenter on the client and Integration Service machines.
You must still configure PowerExchange configuration files on the Integration Service node.
To configure PowerExchange Client for PowerCenter for use on the PowerCenter Integration
Service and Client, you must add NODE statements in the PowerExchange dbmover.cfg file
on the PowerCenter Client and Integration Service machines for those PowerExchange
Listeners to which you wish to connect. See “Modify the PowerExchange Configuration Files”
on page 14.
Read the Release Notes and PowerExchange Migration Guide for any changes to installation
or connectivity.
Upgrading
If you are upgrading from a previous release of PowerCenter, you must perform a repository
upgrade. This process registers the PWXPC plug-in.
If you are upgrading from PowerCenter V8.1.1 SP2 (or any higher Service Pack on V8.1.1)
and are using enhanced restart for CDC session, you must do the following:
1. Prior to migrating to PowerCenter V8.5, cleanly shutdown all CDC sessions and run
recovery on all CDC sessions. PWXPC creates a backup restart token file with a
timestamp appended. Save this file.
2. As a precaution, backup the relational tables which are targets in the CDC sessions. Also
backup the PowerCenter recovery tables.
3. After completing the migration to PowerCenter V8.5, copy the backup restart token files
PWXPC created to the appropriate restart token file for each CDC session. This ensures
that the restart token files contain the restart points from the point of interruption on the
previous release.
4. Cold start the session so PWXPC uses only the newly populated restart token file to
restart the CDC session.
3. Create a node for each PowerExchange Listener you want to register using the following
guidelines:
NODE=(<node name>,TCPIP,<hostname>,<port_number>)
where <node name> is a logical name used to reference the PowerExchange Listener and
<hostname> and <port_number> are the host name (or IP address) and port number of
the PowerExchange Listener.
4. Save your changes.
The following example shows several nodes in a dbmover.cfg file:
/********************************************************************/
/* PowerExchange Configuration File
/********************************************************************/
NODE=(AS400_DB2,TCPIP,AS400_1,2480)
NODE=(MVS1_VSAM,TCPIP,MVS1,5539)
NODE=(MVS2_DB2,TCPIP,10.3.4.5,5538)
For more information, see “Configuration File Parameters” in the PowerExchange Reference
Manual.
7.1.0 7.1.1 7.1.2 7.1.3 7.1.4 7.1.5 8.0.0 8.1.0 8.1.1 8.5.0
5.2.0 No Yes 1 No No No No No No No No
5.2.1 No Yes No No No No No No No No
PowerExchange
1. Support for PWXPC V7.1.1 was introduced with PowerExchange V5.2.0 Patch 02
2. Support for PWXPC V7.1.2 was introduced with PowerExchange V5.2.2 Patch 01
3. Support for PWXPC V7.1.3 was introduced with PowerExchange V5.2.2 Patch 02 is the minimum level required for V71.4 and V7.1.5.
4. See “Using Versions of PowerCenter Earlier than v8.x with PowerExchange v8.x” in the PowerExchange Migration manual.
Chapter 3
17
PWX_Interfaces_for_PowerCenter.book Page 18 Wednesday, October 3, 2007 9:53 AM
Overview
A mapping is a set of source and target definitions linked by transformation objects that
define the rules for data transformation. Mappings represent the data flow between sources
and targets.
Source and target definitions represent metadata for sources and targets. When you create a
source definition, its structure differs depending on the type of source it represents. Non-
relational sources require a multi-group source definition. Relational sources use a single-
group source definition. The source qualifier for a source definition also differs in structure
depending on type of source definition.
After you create a source or target definition, you can include it in a mapping to extract data
from the source or load data to the target. You can extract source data in batch, change, or
real-time mode. For a list of sources and targets that PowerExchange Client for PowerCenter
supports, see Table 1-2 on page 3. This table also lists whether the Integration Service can
read the source data in batch, change, or real-time mode.
With CDC mappings, it is generally necessary to have multiple mappings: a batch mapping to
materialize the target tables from the source tables in preparation for CDC and the CDC
mapping itself which then uses extraction map sources for the source tables. To minimize the
effort in creating these mappings, create any business rules applicable to both the batch and
CDC sessions in Mapplets. For more information about Mapplets, see the PowerCenter
Designer Guide.
This displays the Import from PowerExchange dialog box. The process and the dialog box
displayed is the same for targets.
Figure 3-2 shows the dialog box used to import all PowerExchange sources and targets,
including relational metadata, PowerExchange data maps, and PowerExchange capture
extraction maps:
Figure 3-2. Import from PowerExchange - Source
Additional input fields appear depending upon the “Source Type” chosen. You create the
source and target definitions differently depending on the database type.
After you create a source or target definition, you can edit it.
Note: The Owner name is included in the source definitions for relational metadata and in the
source and target definitions for PowerExchange data maps imported using this dialog. This
information is used, unless overridden, when the source or target is accessed from the
Integration Service node. This eliminates the need to provide the “Owner Name” attribute in
the Session Properties for all source types and “Table Name Prefix” attribute in the Session
Properties for non-relational PowerExchange targets.
DB2 tables that are mapped in PowerExchange as either DB2 data maps or DB2UNLD (DB2
database unload data set) data maps (DB2/390 only) are imported in the same manner as
non-relational data map sources. For more information on how to import these sources, see
“Importing Non-Relational Source Definitions” on page 40.
Use the following procedure to import DB2/390, DB2/400, and DB2/UDB source or target
definitions.
1. To import a DB2 source definition, select Sources > Import from PowerExchange and
select a source type of DB2390, DB2400, or DB2UDB.
To import a DB2 target definition, select Targets > Import from PowerExchange and
select a source type of DB2390, DB2400, or DB2UDB.
The dialog box for a DB2/390 source definition import looks as follows:
The target dialog box looks similar but does not contain the Multi-Record Datamaps and
CDC Datamaps options.
The target dialog box looks similar but does not contain the Multi-Record Datamaps and
CDC Datamaps options.
2. Enter the following connection information:
Required/
Attribute Description
Optional
User Name Required User name for the database you want to connect to.
Multi-Record Datamaps N/a Select to list multi-record data maps. SEQ or VSAM sources only.
CDC Datamaps N/a Select to list CDC extraction maps - source only
Required/
Attribute Description
Optional
Database Name Required Name of the database for connection. DB2400 and DB2UDB only.
Schema Optional Enter a schema name to filter the resulting data maps.
Map name Optional Enter a table name to filter the resulting data maps.
3. Optionally, enter a filter to view particular schemas and tables from the database.
Enter a schema and/or table name. Or, enter a filter condition to display schemas and/or
tables that meet the filter condition. Use one of the following wildcard characters in the
filter condition:
♦ * (asterisk). Represents one or more characters.
♦ ? (question mark). Represents one character.
Use the following syntax when entering a wildcard character:
♦ Enter the filter condition as a prefix. For example, enter A* to display schemas and/or
tables that begin with an A.
♦ Enter the filter condition as a suffix. For example, enter *A to display schemas and/or
tables that end with an A.
♦ Enter the filter condition as a substring. For example, enter *cust* to display schemas
and/or tables that contain “cust.”
Note: Schema and table names are not case sensitive.
4. Click Connect.
The available tables, based on the values specified in the dialog box, will appear in the
Selected Datamaps list.
If no tables are found, “No Data Found” will appear in the Selected Datamaps list.
Schema
Table
1. To import a Microsoft SQL Server source definition, select Sources > Import from
PowerExchange and select a source type of MSSQL.
To import a Microsoft SQL Server target definition, select Targets > Import from
PowerExchange and select a source type of MSSQL.
The dialog box for the Microsoft SQL Server source definition input looks as follows:
The target dialog box looks similar but does not contain the Multi-Record Datamaps and
CDC Datamaps options.
2. Enter the following connection information:
Required/
Attribute Description
Optional
Multi-Record Datamaps N/a Select to list multi-record data maps. SEQ or VSAM sources only.
CDC Datamaps N/a Select to list CDC extraction maps - source only
Required/
Attribute Description
Optional
Schema Optional Enter a schema name to filter the resulting data maps.
Map name Optional Enter a table name to filter the resulting data maps.
3. Optionally, enter a filter to view particular schemas and tables from the database.
Enter a schema and/or table name. Or, enter a filter condition to display schemas and/or
tables that meet the filter condition. Use one of the following wildcard characters in the
filter condition:
♦ * (asterisk). Represents one or more characters.
♦ ? (question mark). Represents one character.
Use the following syntax when entering a wildcard character:
♦ Enter the filter condition as a prefix. For example, enter A* to display schemas and/or
tables that begin with an A.
♦ Enter the filter condition as a suffix. For example, enter *A to display schemas and/or
tables that end with an A.
♦ Enter the filter condition as a substring. For example, enter *cust* to display schemas
and/or tables that contain “cust.”
Note: Schema and table names are not case sensitive.
4. Click Connect.
The available tables, based on the values specified in the dialog box, will appear in the
Selected Datamaps list.
If no tables are found, “No Data Found” will appear in the Selected Datamaps list.
Schema
Table
1. To import an Oracle source definition, select Sources > Import from PowerExchange and
select a source type of Oracle.
To import an Oracle target definition, select Targets > Import from PowerExchange and
select a source type of Oracle.
The Import from PowerExchange > Oracle dialog box looks as follows:
The target dialog box looks similar but does not contain the Multi-Record Datamaps and
CDC Datamaps options.
2. Enter the following connection information:
Required/
Attribute Description
Optional
User Name Required User name for the database connecting to.
Multi-Record Datamaps N/a Select to list multi-record data maps. SEQ or VSAM sources only.
CDC Datamaps N/a Select to list CDC extraction maps - source only.
Required/
Attribute Description
Optional
Schema Optional Enter a schema name to filter the resulting data maps.
Map name Optional Enter a table name to filter the resulting data maps.
3. Optionally, enter a filter to view particular schemas and tables from the database.
Enter a schema and/or table name. Or, enter a filter condition to display schemas and/or
tables that meet the filter condition. Use one of the following wildcard characters in the
filter condition:
♦ * (asterisk). Represents one or more characters.
♦ ? (question mark). Represents one character.
Use the following syntax when entering a wildcard character:
♦ Enter the filter condition as a prefix. For example, enter A* to display schemas and/or
tables that begin with an A.
♦ Enter the filter condition as a suffix. For example, enter *A to display schemas and/or
tables that end with an A.
♦ Enter the filter condition as a substring. For example, enter *cust* to display schemas
and/or tables that contain “cust.”
Note: Schema and table names are not case sensitive.
4. Click Connect.
The available tables, based on the values specified in the dialog box, will appear in the
Selected Datamaps list.
If no tables are found, “No Data Found” will appear in the Selected Datamaps list.
Schema
Table
1. To import an Sybase source definition, select Sources > Import from PowerExchange and
select a source type of Sybase.
To import an Sybase target definition, select Targets > Import from PowerExchange and
select a source type of Sybase.
The Import from PowerExchange > Sybase dialog box looks as follows:
Required/
Attribute Description
Optional
User Name Required User name for the database connecting to.
Multi-Record Datamaps N/A Select to list multi-record data maps. SEQ or VSAM sources only.
CDC Datamaps N/A Select to list CDC extraction maps. N/A for Sybase
Required/
Attribute Description
Optional
Schema Optional Enter a schema name to filter the resulting data maps.
Map name Optional Enter a table name to filter the resulting data maps.
3. Optionally, enter a filter to view particular schemas and tables from the database.
Enter a schema and/or table name. Or, enter a filter condition to display schemas and/or
tables that meet the filter condition. Use one of the following wildcard characters in the
filter condition:
♦ * (asterisk). Represents one or more characters.
♦ ? (question mark). Represents one character.
Use the following syntax when entering a wildcard character:
♦ Enter the filter condition as a prefix. For example, enter A* to display schemas and/or
tables that begin with an A.
♦ Enter the filter condition as a suffix. For example, enter *A to display schemas and/or
tables that end with an A.
♦ Enter the filter condition as a substring. For example, enter *cust* to display schemas
and/or tables that contain “cust.”
Note: Schema and table names are not case sensitive.
4. Click Connect.
The available tables, based on the values specified in the dialog box, will appear in the
Selected Datamaps list.
If no tables are found, “No Data Found” will appear in the Selected Datamaps list.
Schema
Table
A group represents
a table in the
PowerExchange
data map.
Group name
Required/
Attribute Description
Optional
User Name Required User name for the database to connect to.
Multi-Record Datamaps Optional Select to list multi-record data maps. SEQ or VSAM sources only.
Source Type Required Select one of the following source types: ADABAS, DATACOM,
DB2MAP, DB2UNLD, IDMS, IMS, SEQ, or VSAM.
Required/
Attribute Description
Optional
CDC Datamaps n/a Select to list CDC extraction maps - source only.
Schema Optional Enter a schema name to filter the resulting data maps.
Map name Optional Enter a data map name to filter the resulting data maps.
Selected Datamaps n/a Lists the available data maps for the connection, database and filter
details that you entered.
3. Optionally, enter a filter to view particular schemas and tables from the database.
Enter a schema and/or table name. Or, enter a filter condition to display schemas and/or
tables that meet the filter condition. Use one of the following wildcard characters in the
filter condition:
♦ * (asterisk). Represents one or more characters.
♦ ? (question mark). Represents one character.
Use the following syntax when entering a wildcard character:
♦ Enter the filter condition as a prefix. For example, enter A* to display schemas and/or
tables that begin with an A.
♦ Enter the filter condition as a suffix. For example, enter *A to display schemas and/or
tables that end with an A.
♦ Enter the filter condition as a substring. For example, enter *cust* to display schemas
and/or tables that contain “cust.”
Note: Schema and table names are not case sensitive.
4. Click Connect. The available data maps appear in the Selected Datamaps list.
The Designer displays metadata to import. The following two examples show the results
when single record VSAM data maps and multi-record VSAM data maps are selected.
Each record in the multi-record data map will display in Selected Datamaps list when
importing single record data maps. Each record in a multi-record data map is effectively a
single record data map. It is possible to import only a single record within a multi-record
data map as a source.
This example shows the results for single record VSAM data maps:
Figure 3-14. Import from PowerExchange - Single Record Source Data Maps
Schema
Table
This example shows the results for multi-record VSAM data maps:
Figure 3-15. Import from PowerExchange - Multiple Record Source Data Maps
Schema
Table
Required/
Attribute Description
Optional
User Name Required User name for the database to connect to.
Source Type Required Select one of the following source types: ADABAS, IMS, SEQ, or VSAM
Schema Optional Enter a schema name to filter the resulting data maps.
Required/
Attribute Description
Optional
Map name Optional Enter a data map name to filter the resulting data maps.
Selected Datamaps n/a Lists the available data maps for the connection, database and filter
details that you entered.
3. Optionally, enter a filter to view particular schemas and tables from the database.
Enter a schema and/or table name. Or, enter a filter condition to display schemas and/or
tables that meet the filter condition. Use one of the following wildcard characters in the
filter condition:
♦ * (asterisk). Represents one or more characters.
♦ ? (question mark). Represents one character.
Use the following syntax when entering a wildcard character:
♦ Enter the filter condition as a prefix. For example, enter A* to display schemas and/or
tables that begin with an A.
♦ Enter the filter condition as a suffix. For example, enter *A to display schemas and/or
tables that end with an A.
♦ Enter the filter condition as a substring. For example, enter *cust* to display schemas
and/or tables that contain “cust.”
Note: Schema and table names are not case sensitive.
4. Click Connect. The available data maps appear in the Selected Datamaps list. The
Designer displays metadata to import.
The following example shows the results when a Source Type of VSAM is selected:
Figure 3-17. Import from PowerExchange - Non-Relational Target Select Datamaps List
Schema
Table
Table 3-1 describes the attributes the Attributes tab displays for each field in the non-
relational definition:
base_rec Name of the record to which the field belongs. This corresponds to name of the group the field
belongs to in the source definition.
base_fld Name of the base record and table field name in the following format:
<Base_Field_Name>:<Table_Field_Name>
base_fld_offset Offset value from which the field starts. For example, if the value is 5, the field starts at the fifth
position. You determine the offset value of each field based on the order of fields in the data
map.
The following table describes the extensions on the Metadata Extensions tab for a non-
relational definition:
Access Method Method you specified in the data map to access the source database:
- A = Adabas
- D =IMS DL1
- E = VSAM ESDS
- I = IDMS
- K = VSAM KSDS
- O = IMS ODBA
- N = VSAM RRDS
- S = sequential (SEQ)
- W = DB2 unload file (DB2UNLD)
- X = Datacom
- Z = DB2 data map
Schema Name Name of the schema on which you based the data map.
By default, the File Name field contains the value from the PowerExchange data map File
Name field which contains the VSAM data set name or the sequential file name.
Note: Any VSAM and sequential definitions created prior to PowerCenter V8.5 will have a
blank File Name. If desired, this field can be populated manually or by re-importing the data
map.
The following example shows the metadata extensions for a VSAM definition which was
created prior to PowerCenter V8.5:
The following procedure explains how to manually update the File Name field to add or
change the file name in a VSAM or sequential definition:
The Edit Metadata Extension Value panel displays allowing you to enter or change the
file name.
1. To import an extraction map source definition, select Sources > Import from
PowerExchange and select the CDC Datamaps box and then select the desired source
type.
The dialog box and parameters displayed are the same for each non-relational source
type.
Required/
Attribute Description
Optional
User Name Required User name for the database connecting to.
Multi-Record Datamaps N/a Select to list multi-record data maps. SEQ or VSAM sources only.
Source Type Required Select one of the following source types: ADABAS, DATACOM,
DB2390, DB2400, DB2UDB, IDMS, IMS, MSSQL, ORACLE or VSAM.
CDC Datamaps Required Select to list CDC extraction maps - source only.
Schema Optional Enter a schema name to filter the resulting data maps.
Map name Optional Enter a data map name to filter the resulting data maps.
2. Optionally, enter a filter to view particular schemas and tables from the database.
Enter a schema and/or table name. Or, enter a filter condition to display schemas and/or
tables that meet the filter condition. Use one of the following wildcard characters in the
filter condition:
♦ * (asterisk). Represents one or more characters.
♦ ? (question mark). Represents one character.
Use the following syntax when entering a wildcard character:
♦ Enter the filter condition as a prefix. For example, enter A* to display schemas and/or
tables that begin with an A.
♦ Enter the filter condition as a suffix. For example, enter *A to display schemas and/or
tables that end with an A.
♦ Enter the filter condition as a substring. For example, enter *cust* to display schemas
and/or tables that contain “cust.”
Note: Schema and table names are not case sensitive.
3. Click Connect. The Designer displays the metadata to import. The extraction maps
shown will be filtered based on the source type specified.
Schema
Table
In this example, only extraction maps for a specific Schema and Map name appear in the
Designer.
4. Select the data map you want to import.
♦ Hold down the Shift key to select blocks of tables.
♦ Hold down the Ctrl key to make non-contiguous selections within a data map.
♦ Use the Select all button to select all tables.
♦ Use the Select none button to clear all highlighted selections.
5. Click OK.
The source definition appears in the workspace. The Designer uses the data map name as
the name of the source definition.
Table 3-3 describes the attributes the Attributes tab displays for each field in the source
definition:
base_rec blank.
base_fld blank
base_fld_tpe blank.
base_fld_offset blank
The following table describes the extensions on the Metadata Extensions tab for an extraction
map definition:
Access Method Method you specified in the data map to access the source database:
- A = Adabas
- B = DB2/390 and DB2/400
- D =IMS
- E = VSAM ESDS
- I = IDMS
- K = VSAM KSDS
- L = MSSQL
- N = VSAM RRDS
- P = Oracle
- V = DB2/UDB
- X = Datacom
Original Name Original table name in relational database or PowerExchange data map
Original Schema Original schema or owner name in relational database or PowerExchange data map
You must be able to connect to PowerExchange on the source or target database platform to
preview data.
The contents of the table appear in the Preview Data dialog box.
8. To change the number of rows you want to preview, enter a new number and click
Refresh.
9. Click Close to exit.
The contents of the table appear in the Preview Data dialog box.
The time out value indicates the maximum time (in seconds) to wait for additional data
at the end of log. After this time period expires, EOF is returned and the number of rows
requested is displayed. Returning EOF terminates any further retrieval of data from the
change stream.
The default is 10 seconds. The value can be any number between 0 and 86400. A value of
0 indicates that EOF is returned as soon as the end of log is reached whereas 86400
indicates that EOF is never returned. Do not specify 86400 as no data is displayed and
the Preview Data session hangs until the extraction task in PowerExchange Listener is
stopped.
8. Enter the number of rows you want to preview.
The default is 10. The Preview Data dialog box can display up to 500 rows and up to
65,000 columns.
9. Click Connect.
The contents of the table appear in the Preview Data dialog box.
11. To change the number of rows you want to preview, enter a new number and click
Connect.
12. Click Close to exit.
PowerExchange
data map table name
The PowerExchange NRDB Batch application connection is used to read the data and uses
Group Source functionality. Group Source processing for multi-record non-relational data
maps is done for each source definition. Each source definition in a mapping will result in a
connection to PowerExchange to read the source data. It is possible to have multiple multi-
record source definitions in a mapping and each one can use Group Source processing.
For non-relational CDC sources, the source metadata must be imported using PWXPC.
Tip: Use extraction maps, or CDC Datamaps, for CDC sources as this eliminates the need to
specify the extraction map name in the Session Properties. It also eliminates the need to add
any of the special DTL columns: the DTL__CAPX fields, the DTL__CI change indicator
fields, and the DTL__BI before image fields. This can significantly simplify the mapping
design process.
Group Source functionality is invoked for each source type. A mapping should contain only a
specific source type. Sessions with a mappings containing multiple source types, even if the
same change stream is being read, will fail with:
PWXPC_10080 [ERROR] All the readers should be of one database type only
For example, a mapping containing both VSAM and IMS sources will fail with the 10080
message. Two separate mappings should be made: one for the VSAM sources and one for the
IMS sources. If these two mappings are included in two sessions in the same workflow, they
result in separate Group Source connections to the change stream: one for VSAM and one for
IMS.
Group Source for batch access to non-relational sources requires that the source be imported
as a multi-record data map and is used for an individual source definition. With CDC access,
group source is invoked at a mapping level for all source definitions rather than at an
individual source definition level.
The invocation of Group Source occurs automatically when a PWX Change or Real-Time
connection is used in a session, regardless of the number of sources included in the session. It
also occurs automatically if a multi-record source definition exists in a mapping.
The following diagram shows an example of a mapping for three DB2 sources:
When you include this mapping in a session that uses the PowerExchange DB2 CDC
application connection, PowerExchange reads through the change stream a single time, using
Group Source, to extract the changes for all three source tables. The changes for each source
are provided to the source qualifier in the chronological order in which each unit of work
(UOW) completed.
When you include this mapping in a session that uses a PowerExchange DB2 relational
connection, PowerExchange reads each source table separately. A unique pipeline is created
for each source which results in three unique tasks in the PowerExchange Listener.
Transformation Datatypes
The transformation datatypes in Source Qualifier and Application Multi-Group Source
Qualifier transformations are internal datatypes based on ANSI SQL-92 generic datatypes,
which PowerCenter uses to move data across platforms. When the Integration Service reads
data from a source, it converts the data from the PowerExchange data type to the
transformation data type. When you run a session, the Integration Service performs
transformations based on the transformation datatypes. When writing data to a target, the
Integration Service converts the data based on the datatypes in the target definition.
The transformation data type for all ports in the Application Multi-Group Source Qualifier
transformation are predefined. You cannot change the data type for any of the fields in the
Application Multi-Group Source Qualifier transformations. For more information about
transformation datatypes, see the PowerCenter Designer Guide.
stream is maintained until the changes reach the Writer. This then ensures that any CDC
data placed into a dynamic lookup cache is accessible to lookups sharing that cache in other
pipelines.
Warning: The use of these custom properties will impact session throughput as it will single-
thread all source data from the Reader through to the Writer. As a result, these custom
properties should only be specified when there is a need to share CDC data stored in a
dynamic cache across pipeline.
Chapter 4
PWXPC - Configuring
Connections
This chapter includes the following topics:
♦ Overview, 78
♦ DB2 Batch Mode Relational Database Connections, 84
♦ DB2 CDC Mode Application Connections, 87
♦ NRDB Batch Mode Application Connections, 90
♦ NRDB CDC Mode Application Connections, 91
♦ NRDB Lookup Relational Connections, 94
♦ MSSQL Batch Mode Relational Connections, 95
♦ MSSQL CDC Mode Application Connections, 97
♦ Oracle Batch Mode Relational Connections, 100
♦ Oracle CDC Mode Application Connections, 102
♦ Sybase Batch Mode Relational Connections, 105
♦ Configuring Connections Attributes, 106
♦ Understanding Commit Processing with PWXPC, 121
77
PWX_Interfaces_for_PowerCenter.book Page 78 Wednesday, October 3, 2007 9:53 AM
Overview
Before the Integration Service can access a source or target in a session, you must configure
connections in the Workflow Manager. When you create or modify a session that reads from
or writes to a database of file, you can select only configured source and target databases.
Connections are saved in the repository.
For PowerExchange Client for PowerCenter, you configure relational database or application
connections, depending upon the source or the target type.
Overview 79
PWX_Interfaces_for_PowerCenter.book Page 80 Wednesday, October 3, 2007 9:53 AM
Configuring Connections
The connection you configure depends upon the database or data structure.
Source and target connections can be configured for:
♦ Extracting data (Batch) from relational or non-relational sources.
♦ Extracting changed data (Change or Real-Time) from non-relational or relational sources.
♦ Loading data (Batch) to a relational target.
♦ Loading data (Batch) to a non-relational target.
For more information about available connection types, see Table 4-1 on page 78 and
Table 4-2 on page 79.
To configure connections:
Configuring Connections 81
PWX_Interfaces_for_PowerCenter.book Page 82 Wednesday, October 3, 2007 9:53 AM
In the Select Type field, select the type of connection you want to create. For a list of
connections to configure according to data source and extraction mode, see Table 4-1 on
page 78. For a list of connections to configure according to target data type, see Table 4-
2 on page 79.
3. Click New.
The Connection Object Definition dialog box appears. The relational and application
dialog boxes are very similar. An application dialog box is shown here:
4. Enter the values for the connection attributes. The various connection types are
described in subsequent sections in this chapter.
5. Click OK.
The new connection appears in the Application or Relational Object Browser.
To edit or delete a relational database or application connections, select the connection from
the list and click the appropriate button.
Configuring Connections 83
PWX_Interfaces_for_PowerCenter.book Page 84 Wednesday, October 3, 2007 9:53 AM
Table 4-3. PWX DB2390, DB2400, and DB2UDB Relational Database Connection Attributes
DB2390
Connection
Required DB2400 Description
Attribute
DB2UDB
Code Page Yes All Select the code page for the Integration Server to use to extract
data from the source database.
Note: In Unicode mode, PWXPC sets the code page with this value
overriding any code page specification in the PowerExchange
configuration file.
Location Yes All Location of the source or target database as specified as a node in
the PowerExchange configuration file dbmover.cfg.
User Name Yes All User name for the database connected to.
Database file No DB2400 Overrides any occurrence of the specified filename (in any SQL
overrides statement) with the library/filename/member specified, regardless
of whether the file is qualified or not. This includes any files
qualified using Library List.
Separate libraries with semicolons.
Note: If both Library List and Database file overrides are specified
and a table exists both, the Database file overrides takes
precedence.
Library List No DB2400 List of libraries that PowerExchange searches to qualify the table
name for Select, Insert, Delete Or Update statements.
PowerExchange searches the list if the table name is unqualified.
Separate libraries with semicolons.
Note: If both Library List and Database file overrides are specified
and a table exists both, the Database file overrides takes
precedence.
Compression No All Select to compress source data during the PowerCenter session.
See “Configuring Encryption and Compression” on page 106.
Table 4-3. PWX DB2390, DB2400, and DB2UDB Relational Database Connection Attributes
DB2390
Connection
Required DB2400 Description
Attribute
DB2UDB
Interpret as Rows No All Specifies whether, or not, pacing size is in number of rows.
See “Configuring Pacing” on page 107.
Bulk Load* No DB2/390 Select to cause PowerExchange to load data to DB2/390 targets
using the DB2 bulk loader. If you select this option, you can
configure the remaining connection attributes. Otherwise, the
PowerExchange ignores these attributes.
Filename* No DB2/390 Enter a file name to use as a stem for files created on the host
machine.
This can be overridden in the Workflow. If using this connection for
a DB2 Bulk Load may load DB2 partitions. When loading partitions
it is easiest to use a single connection but to change the file name
in the targets.
Space* Yes DB2/390 Enter one of the following values to allocate MVS space in tracks
or cylinders:
- TRACK
- CYLINDER
Default is TRACK.
Primary Space* Yes DB2/390 Value for the primary space on MVS. Default is 0.
Secondary Space* Yes DB2/390 Value for the secondary space on MVS. Default is 0.
JCL Template* Yes DB2/390 Name of the JCL file template on the target system.
Default is DB2LDJCL.
CTL Template* Yes DB2/390 Name of the Control file template on the target system.
Default is DB2LDCTL.
Table 4-3. PWX DB2390, DB2400, and DB2UDB Relational Database Connection Attributes
DB2390
Connection
Required DB2400 Description
Attribute
DB2UDB
Time* Yes DB2/390 Value from 1 to 99998 to specify the wait time in seconds when you
select JOB for the Mode Type and TIMED for Mode Time.
Default is 0.
Write Mode No All Select the write mode. See “Configuring Write Mode” on page 109.
Default is Confirm Write On.
Reject File No All Overrides the default prefix of PWXR for the reject file.
PowerExchange creates the reject file on the target machine when
the Write Mode is Asynchronous with Fault Tolerance.
Table 4-4. DB2390, DB2400, and DB2UDB CDC Mode Application Connection Attributes
Connection Change or
Required Description
Attribute Real-time
Code Page Yes Both Select the code page for the Integration Service to use to extract
data from the source database.
Note: In Unicode mode, PWXPC sets the code page with this
value overriding any code page specification in the
PowerExchange configuration file.
Location Yes Both Location of the source or target database as specified as a node
in the PowerExchange configuration file dbmover.cfg.
Interpret as Rows No Both Specifies whether, or not, pacing size is in number of rows.
See “Configuring Pacing” on page 107.
Table 4-4. DB2390, DB2400, and DB2UDB CDC Mode Application Connection Attributes
Connection Change or
Required Description
Attribute Real-time
Number of Runs to No Both Specify the maximum number of backup copies to keep of the
Keep RestartToken Restart Token File.
File See “Managing Session Log and Restart Token File History” on
page 172.
Default is 0.
Recovery Cache No Both Specify file cache folder to enable recovery for the session. See
Folder “Enabling Session Recovery” on page 163.
Default is $PMRootDir/Cache.
UOW Count No Both Specifies the number of units of work (UOWs) you want the
PWXPC to read from the source before flushing data to the
target.
If you enter:
-1 = UOW count is not used
0 = UOW count is not used
n = n is the count of UOWs
Reader Time Limit No Real Time Specifies the number of seconds that Integration Service reads
data from the source before stopping.
If you enter 0, Reader Time Limit does not limit the reader time.
This attribute is intended for testing purposes only.
Tip: Use Idle Time instead of Reader Time Limit.
Default is 0.
Idle Time No Real Time Specifies the number of seconds the PowerExchange Listener
remains idle after reaching the end of the change log (as
indicated by message PWX-09967) before returning an end-of-
file (EOF).
If you enter:
-1 = EOF is never retuned; the session runs continuously.
0 = EOF is returned at the end of log; the session terminates
successfully.
n = n is the number of seconds.
Table 4-4. DB2390, DB2400, and DB2UDB CDC Mode Application Connection Attributes
Connection Change or
Required Description
Attribute Real-time
Real-time Flush No Real Time Specifies the milliseconds between buffer flushes.
Latency Valid values are between 0 and 86400 milliseconds. PWXPC
sets values between 0 and 2000 to 2000.
See “Configuring Real-Time Flush Latency” on page 117.
Default is 0.
Commit Threshold No Real Time Specifies the number of change records (not UOWs) after which
a commit should be inserted into the change stream.
See “Configuring Commit Threshold” on page 119.
Default is 0.
Journal Name* No Both Overrides the library and journal name in the PowerExchange
CAPI_CONNECTION.
Specify complete library and journal names in the format:
library/journal
Library/File No Both Overrides the library and file name in the extraction map.
Override* Specify complete library and file names in the format:
library/file
Event Table No Real Time Specifies the PowerExchange extraction map name used for
event processing.
See “Configuring Event Table Processing” on page 112..
CAPI Connection No Real Time Overrides the default CAPI connection name.
Name Override See “Configuring the CAPI Connection Name Override” on
page 113.
Retrieve PWX Log No Both Includes all related PowerExchange log entries in the session
Entries log.
See “Retrieving PWX Log Entries” on page 110.
* These attributes only apply to PWX CDC DB2400 Real Time application connections.
Connection
Required Description
Attribute
Code Page Yes Select the code page for the Integration Service to use to extract data from the
source database.
Note: In Unicode mode, PWXPC sets the code page with this value overriding any
code page specification in the PowerExchange configuration file.
Location Yes Location of the source or target database as specified as a node in the
PowerExchange configuration file dbmover.cfg.
User Name Yes User name for the database being connected to.
Compression No Select to compress source data during the PowerCenter session. See
“Configuring Encryption and Compression” on page 106.
Confirm Write No Selects the write mode. See “Configuring Write Mode” on page 109.
Default is Confirm Write On.
Retrieve PWX Log No Include all related PowerExchange log entries in the session log.
Entries
Connection Change or
Required Description
Attribute Real-time
Code Page Yes Both Select the code page for the Integration Service to use to extract
data from the source database.
Note: In Unicode mode, PWXPC sets the code page with this
value overriding any code page specification in the
PowerExchange configuration file.
Location Yes Both Location of the source or target database as specified as a node
in the PowerExchange configuration file dbmover.cfg.
User Name Yes Both User name for the database being connected to.
Interpret as Rows No Both Specifies whether, or not, pacing size is in number of rows.
See “Configuring Pacing” on page 107.
Connection Change or
Required Description
Attribute Real-time
Number of Runs to No Both Specify the maximum number of backup copies to keep of the
Keep RestartToken Restart Token File.
File See “Managing Session Log and Restart Token File History” on
page 172.
Default is 0.
Recovery Cache No Both Specify file cache folder to enable recovery for the session. See
Folder “Enabling Session Recovery” on page 163.
Default is $PMRootDir/Cache.
UOW Count No Both Specifies the number of units of work (UOWs) you want the
PWXPC to read from the source before flushing data to the
target.
If you enter:
-1 = UOW count is not used
0 = UOW count is not used
n = n is the count of UOWs
Reader Time Limit No Real Time Specifies the number of seconds that Integration Service reads
data from the source before stopping.
If you enter 0, Reader Time Limit does not limit the reader time.
This attribute is intended for testing purposes only.
Tip: Use Idle Time instead of Reader Time Limit.
Default is 0.
Idle Time No Real Time Specifies the number of seconds the PowerExchange Listener
remains idle after reaching the end of the change log (as
indicated by message PWX-09967) before returning an end-of-
file (EOF).
If you enter:
-1 = EOF is never retuned; the session runs continuously.
0 = EOF is returned at the end of log; the session terminates
successfully.
n = n is the number of seconds.
Connection Change or
Required Description
Attribute Real-time
Real-time Flush No Real Time Specifies the milliseconds between buffer flushes.
Latency Valid values are between 0 and 86400 milliseconds. PWXPC
sets values between 0 and 2000 to 2000.
See “Configuring Real-Time Flush Latency” on page 117.
Default is 0.
Commit Threshold No Real Time Specifies the number of change records (not UOWs) after which
a commit should be inserted into the change stream.
See “Configuring Commit Threshold” on page 119.
Default is 0.
Event Table No Real Time Specifies the PowerExchange extraction map name used for
event processing.
See “Configuring Event Table Processing” on page 112.
CAPI Connection No Real Time Overrides the default CAPI connection name.
Name Override See “Configuring the CAPI Connection Name Override” on
page 113
Retrieve PWX Log No Both Include all related PowerExchange log entries in the session log.
Entries
Connection
Required Description
Attribute
Code Page Yes Select the code page for the Integration Service to use to extract data from the
source database.
Note: In Unicode mode, PWXPC sets the code page with this value overriding any
code page specification in the PowerExchange configuration file.
Location Yes Location of the source or target database as specified as a node in the
PowerExchange configuration file dbmover.cfg.
User Name Yes User name for the Listener being connected to.
Compression No Select to compress source data during the PowerCenter session. See
“Configuring Encryption and Compression” on page 106.
Connection
Required Description
Attribute
Code Page Yes Select the code page for the Integration Service to use to extract data from the
source database.
Note: In Unicode mode, PWXPC sets the code page with this value overriding any
code page specification in the PowerExchange configuration file.
Location Yes Location of the source or target database as specified as a node in the
PowerExchange configuration file dbmover.cfg.
Database Name Yes Name of the database in the SQL Server instance.
User Name Yes User name for the database being connected to.
Connection
Required Description
Attribute
Write Mode No Select the write mode. See “Configuring Write Mode” on page 109.
Default is Confirm Write On.
Connection
Required Description
Attribute
Code Page Yes Select the code page for the Integration Service to use to extract data from the
source database.
Note: In Unicode mode, PWXPC sets the code page with this value overriding
any code page specification in the PowerExchange configuration file.
Location Yes Location of the source or target database as specified as a node in the
PowerExchange configuration file dbmover.cfg.
User Name Yes User name for the Microsoft SQL Server database.
Compression No Select to compress source data during the PowerCenter session. See
“Configuring Encryption and Compression” on page 106.
Connection
Required Description
Attribute
Number of Runs to No Specify the maximum number of backup copies to keep of the Restart Token
Keep RestartToken File.
File See “Managing Session Log and Restart Token File History” on page 172.
Default is 0.
Recovery Cache No Specify file cache folder to enable recovery for the session. See “Enabling
Folder Session Recovery” on page 163.
Default is $PMRootDir/Cache.
UOW Count No Specifies the number of units of work (UOWs) you want the PWXPC to read
from the source before flushing data to the target.
If you enter:
-1 = UOW count is not used
0 = UOW count is not used
n = n is the count of UOWs
Reader Time Limit No Specifies the number of seconds that Integration Service reads data from the
source before stopping.
If you enter 0, Reader Time Limit does not limit the reader time. This attribute
is intended for testing purposes only.
Tip: Use Idle Time instead of Reader Time Limit.
Default is 0.
Idle Time No Specifies the number of seconds the PowerExchange Listener remains idle
after reaching the end of the change log (as indicated by message PWX-
09967) before returning an end-of-file (EOF).
If you enter:
-1 = EOF is never retuned; the session runs continuously.
0 = EOF is returned at the end of log; the session terminates successfully.
n = n is the number of seconds.
Commit Threshold No Specifies the number of change records (not UOWs) after which a commit
should be inserted into the change stream.
See “Configuring Commit Threshold” on page 119.
Default is 0.
Connection
Required Description
Attribute
Event Table No Specifies the PowerExchange extraction map name used for event processing.
See“Configuring Event Table Processing” on page 112.
Retrieve PWX Log No Include all related PowerExchange log entries in the session log.
Entries
Connection
Required Description
Attribute
Code Page Yes Select the code page for the Integration Service to use to extract data from the
source database.
Note: In Unicode mode, PWXPC sets the code page with this value overriding any
code page specification in the PowerExchange configuration file.
Location Yes Location of the source or target database as specified as a node in the
PowerExchange configuration file dbmover.cfg.
User Name Yes User name for the database being connected to.
Compression No Select to compress source data during the PowerCenter session. See
“Configuring Encryption and Compression” on page 106.
Connection
Required Description
Attribute
Write Mode No Select the write mode. See “Configuring Write Mode” on page 109.
Default is Confirm Write On.
Reject File No Overrides the default prefix of PWXR for the reject file.
PowerExchange creates the reject file on the target machine when the Write Mode
is Asynchronous with Fault Tolerance.
Connection Change or
Required Description
Attribute Real-time
Code Page Yes Both Select the code page for the Integration Service to use to extract
data from the source database.
Note: In Unicode mode, PWXPC sets the code page with this
value overriding any code page specification in the
PowerExchange configuration file.
Location Yes Both Location of the source or target database as specified as a node
in the PowerExchange configuration file dbmover.cfg.
Interpret as Rows No Both Specifies whether, or not, pacing size is in number of rows.
See “Configuring Pacing” on page 107.
Connection Change or
Required Description
Attribute Real-time
Number of Runs to No Both Specify the maximum number of backup copies to keep of the
Keep RestartToken Restart Token File.
File See “Managing Session Log and Restart Token File History” on
page 172.
Default is 0.
Recovery Cache No Both Specify file cache folder to enable recovery for the session. See
Folder “Enabling Session Recovery” on page 163.
Default is $PMRootDir/Cache.
UOW Count No Both Specifies the number of units of work (UOWs) you want the
PWXPC to read from the source before flushing data to the
target.
If you enter:
-1 = UOW count is not used
0 = UOW count is not used
n = n is the count of UOWs
Reader Time Limit No Real Time Specifies the number of seconds that Integration Service reads
data from the source before stopping.
If you enter 0, Reader Time Limit does not limit the reader time.
This attribute is intended for testing purposes only.
Tip: Use Idle Time instead of Reader Time Limit.
Default is 0.
Idle Time No Real Time Specifies the number of seconds the PowerExchange Listener
remains idle after reaching the end of the change log (as
indicated by message PWX-09967) before returning an end-of-
file (EOF).
If you enter:
-1 = EOF is never retuned; the session runs continuously.
0 = EOF is returned at the end of log; the session terminates
successfully.
n = n is the number of seconds.
Connection Change or
Required Description
Attribute Real-time
Real-time Flush No Real Time Specifies the milliseconds between buffer flushes.
Latency Valid values are between 0 and 86400 milliseconds. PWXPC
sets values between 0 and 2000 to 2000.
See “Configuring Real-Time Flush Latency” on page 117.
Default is 0.
Commit Threshold No Real Time Specifies the number of change records (not UOWs) after which
a commit should be inserted into the change stream.
See “Configuring Commit Threshold” on page 119.
Default is 0.
Instance Name No Real Time Overrides the instance information in the ORACLEID statement
in the PowerExchange configuration file (dbmover.cfg).
For more information, see the PowerExchange Oracle Adapter
Guide.
Connect String No Real Time Overrides the connection value in the ORACLEID statement in
the PowerExchange configuration file (dbmover.cfg).
For more information, see the PowerExchange Oracle Adapter
Guide.
Source Schema No Both Overrides the schema name for a group of registrations.
Name For more information, see the PowerExchange Oracle Adapter
Guide.
Event Table No Real Time Specifies the PowerExchange extraction map name used for
event processing.
See “Configuring Event Table Processing” on page 112..
CAPI Connection No Real Time Overrides the default CAPI connection name.
Name Override See “Configuring the CAPI Connection Name Override” on
page 113
Retrieve PWX Log No Both Include all related PowerExchange log entries in the session log.
Entries
Connection
Required Description
Attribute
Code Page Yes Select the code page for the Integration Service to use to extract data from the
source database.
Note: In Unicode mode, PWXPC sets the code page with this value overriding any
code page specification in the PowerExchange configuration file.
Location Yes Location of the source or target database as specified as a node in the
PowerExchange configuration file dbmover.cfg.
User Name Yes User name for the database being connected to.
Compression No Select to compress source data during the PowerCenter session. See
“Configuring Encryption and Compression” on page 106.
Write Mode No Select the write mode. See “Configuring Write Mode” on page 109.
Default is Confirm Write On.
Encryption Type Yes Select one of the following values for the encryption type:
- None
- RC2
- DES
Default is None.
Encryption Level Yes If you select None for Encryption Type, then PowerExchange ignores this
value.
If you select RC2 or DES for Encrypt, enter one of the following values for
the encryption level:
Default is 1.
Configuring Pacing
You can configure the pacing size to slow the data transfer rate from the PowerExchange
Listener. The pacing size determines the amount of data the PowerExchange Listener passes
to the source or target. Configure the pacing size if an external application, database, or the
Integration Service node is a bottleneck during the session. For more information about
pacing size, see the PowerExchange Reference Manual.
Table 4-14 describes the pacing attributes:
Pacing Size No Enter the amount of data the source system can pass to the
PowerExchange Listener. The lower the value, the faster the session
performance.
Default is 0.
Interpret as Rows No Select to represent the pacing size in number of rows. If you clear this
option, the pacing size represents kilobytes.
The application that processes this field uses the x’00’ as a delimiter and parses the field into
three strings:
♦ x’C1C2C3’ (ABC)
♦ x’C4C5C6C7’ (DEFG)
♦ x’C8C9’ (HI)
If this field is read from PowerExchange by PowerCenter, the result be only the string ABC.
The rest of the field would be truncated when the first null indicator is detected in the data.
This connection attributes exists to allow these types of fields to be extracted. If selected,
embedded null indicators (x’00’) are converted to spaces (x’40’). As a result, the example
above would result in the field containing the following hexadecimal EBCDIC data:
C1C2C340C4C5C6C740C8C9
Convert character data to No Select to convert embedded nulls in character fields to spaces.
string
Default is to leave embedded nulls as-is.
Write Mode No - Confirm Write On sends data to the PowerExchange Listener and waits
for a success/no success response before sending more data. This is
important if good error recovery is a priority. It has the drawback of
slowing data transfer rates.
- Confirm Write Off sends data to the PowerExchange Listener without
waiting for a success/no success response (asynchronously). While
faster than Confirm Write On, only use this option when the target table
can be reloaded if an error occurs.
- Asynchronous with Fault Tolerance sends data to the PowerExchange
Listener asynchronously similar to Confirm Write Off but with the ability to
detect errors. This provides the speed of Confirm Write Off with the data
integrity of Confirm Write On.
Reject File No Overrides the default prefix of PWXR for the reject file.
PowerExchange creates the reject file on the target machine when the
Write Mode is Asynchronous with Fault Tolerance.
Asynchronous (write) with Fault Tolerance combines the speed of Confirm Write Off with
error detection of Confirm Write One. Data is buffered and sent asynchronously to the
PowerExchange Listener. A reject file is created on the target machine when SQL errors occur
allowing any errors to be corrected without reloading the entire table. You can also specify
how to handle specific SQL return codes. In order to stop session execution when errors are
encountered, specify a value larger than 0 in the Session Error handling option “Stop on
errors” on the Config Object tab. See the PowerExchange Reference Manual for a complete
description of Asynchronous Write with Fault Tolerance.
Retrieve PWX Log Entries No Include all related PowerExchange log entries in the session log.
Default is no PowerExchange messages in the session log.
Retrieving PowerExchange log entries will include into the PowerCenter session log messages
related to the session that are normally found only in the PowerExchange log. This allows a
single log to provide a view of both PowerCenter and PowerExchange processing, speeding
diagnosis when errors occur. The PowerExchange messages related to the session are returned
in the session log as a part of message PWXPC_10091.
Default is BA.
You can configure whether before-image data is extracted for update operations using the
Image Type specification. PowerExchange captures before and after image data for all
updates, regardless of source type. The before-image data can always be extracted in real-time
mode. In change mode, it is possible that only after-image data is available if the changes have
been specifically condensed with only after-images. See the appropriate PowerExchange
Adapter Guide for the source type for additional information the change Condense process.If
you specify Image Type=BA, the before-image and after-image data of the entire row that was
updated are presented as two separate rows: a delete with the before-image data and an insert
with the after-image data.
If you specify Image Type=AI, then only after-image data is provided for update records
(unless you explicitly request before-image data). With AI processing, updates are passed as
update records and not changed to a delete/insert pair as occurs in BA processing.
It is possible, selectively by column, to request that before-image column data be embedded
within the after-image update record. When this form of before-image data is used, the
change remains an update (as opposed to being changed into a delete/insert pair). When you
use embedded before-image columns, you should specify AI for Image Type.
In order to request that the before-image of the column be embedded into the update row,
you must alter the PowerExchange extraction map. In the PowerExchange Navigator, select
the columns for which you would like before-image data. This will create before-image
columns (DTL__BI_columnname) within the extraction map for those columns selected. This
allows the before-image data to be easily manipulated in your mapping as it is contained in
the same update record as the after-image data.
One possible use for embedded before-image data is to handle update records where the
primary key has been updated. In some relational databases (such as DB2/390), it is possible
to do an update to the primary key (thereby changing the key value). The RDBMS
understands that this operation is equivalent to deleting the row and then re-adding it with a
new primary key. This activity is logged as an update and so will be passed as an update
record when extracted. In some circumstances this may causes problems when attempting to
apply this update to the target database as some relational databases does not allow primary
key values to be updated. Including the before-image data for key columns will allow this type
of activity to be detected. The Flexible Key Custom Transformation will allow this to be
properly handled at the target.
Note: To use the Flexible Key Custom Transformation, you must configure before-image
columns in the PowerExchange extraction map. For more information on the Flexible Key
Custom Transformations, see “Flexible Key Custom Transformation” on page 175.
For additional information about configuring before-image columns, see the PowerExchange
Adapter Guide for the source type.
Connection Change
Required Description
Attribute Real Time
Event Table No Real Time Specifies the PowerExchange extraction map name used for event
table processing.
Using an event table, you can stop real-time CDC sessions based on an external event. For
example, you want to stop a CDC extraction every night after all of the day’s changes have
been processed. You can use an event table to do this by making a change to that table at
midnight. When PowerExchange processes the change for the event table, it will stop reading
changes at that point and shutdown the extraction.
To use event table processing, complete the following tasks:
1. Register the event table for CDC.
The event table must be the same source type and on the same machine as the CDC data
being extracted. For example, if you are extracting DB2 changes from MVS, then the
event table must be a DB2 table in the same DB2 subsystem as the DB2 changes.
2. Specify the extraction map name for the event table in the connection Event Table
attribute for the CDC sessions you wish to stop based on an event.
3. When the event occurs, make a change to the event table.
4. When PowerExchange reads the change to the event table, it places an end-of-file (EOF)
in the change stream.
5. PWXPC processes the EOF, passes it along to the Integration Service and shuts down the
reader.
6. The Integration Service completes writing all of the data currently in the pipeline and
ends the session.
Connection Change
Required Description
Attribute Real Time
CAPI Connection No Real Time Overrides the default CAPI connection name.
Name Override
Connection Change
Required Description
Attribute Real-time
Idle Time No Real Time Specifies the number of seconds the PowerExchange
Listener remains idle after reaching the end of the change
log (as indicated by message PWX-09967) before returning
an end-of-file (EOF).
If you enter:
Use the Idle Time session condition to indicate whether the real-time session should run
continuously (forever) or shutdown after a specified period of time. This parameter requires a
valid value and has a valid default value.
The Idle Time timing starts when the PowerExchange Listener begins reading changed data
for the source(s).
If -1 is entered for Idle Time, PowerExchange will never return an end-of-file (EOF) to the
Integration Server thereby causing the session to run forever. This is generally how a real-time
session is setup. It is the default value primed for Idle Time in all of the real-time
connections.
Continuous extraction sessions must be stopped using either the PowerExchange STOPTASK
command or through PowerCenter using Workflow Monitor Stop/Abort or the pmcmd
commands to stop and abort tasks and workflows.
♦ If you stop the session or workflow using the PowerCenter Workflow Monitor or using
pmcmd, this is considered a normal termination. PowerCenter will perform a graceful
stop, instructing the CDC reader and the writers to shutdown and waiting until all data
currently in the pipeline is processed. For more information about stopping real-time
sessions and workflows, see the PowerCenterWorkflow Administration Guide.
♦ If you abort the session or workflow using the PowerCenter Workflow Monitor or using
pmcmd, this is considered an abnormal termination since PowerCenter does not wait for
the reader and writer to shutdown or until all data in the pipeline is processed. For more
information about aborting sessions and workflows, see the PowerCenterWorkflow
Administration Guide.
♦ The PowerExchange STOPTASK command will stop the extraction task in the
PowerExchange Listener and pass an EOF to the Integration Server which will terminate
the session successfully. For more information on the PowerExchange STOPTASK
command, see the Task Utility (DTLUTSK) documented in the PowerExchange Utilities
manual.
Warning: Ensure that you have switched the Session Properties “Commit Type” attribute to
Source and unchecked the “Commit at End of File” attribute. By default, “Commit at End of
File” is on and it will cause data to be committed after the CDC reader has shutdown and
committed the restart tokens. As a result, when the session is restarted, duplicate data will be
sent to the target.
If 0 is entered for Idle Time, PowerExchange will return an EOF to the Integration Service
when the end-of-log (EOL) is reached. After the EOF is received, the Integration Service will
terminate the session successfully, meaning that all data will be committed and the restart
token file will be updated. The end-of-log is determined by what was the current end of the
change stream at the point that PowerExchange started to read the change stream. This
concept of EOL is required because the change stream is generally not static so the actual
“end-of ’-log” is continually moving forward. PowerExchange issues the following message
when EOL is reached:
PWX-09967 CAPI i/f: End of log for time <yy/mm/dd> <hh:mm:ss> reached
For example, if a session starts reading a change stream at 10:00 a.m., the EOL at that point is
determined. After PowerExchange reaches that point in the change stream, it will return EOF
to the Integration Service. This means that changes recorded in the change stream after 10:00
a.m. will not be processed. Specifying 0 for Idle time is a useful in situations where you want
to extract changed data for sources periodically as opposed to continuously.
If a positive number is specified for Idle Time, then the session will run until no data is
returned for the period of time specified. After Idle Time is reached, PowerExchange will
send an end-of-file (EOF) to the Integration Service and the session will terminate
successfully. Specifying a low Idle Time (1 for example) can result in this time being reached
before all available data in the change stream has been read.
The following message is issued to indicate that the Idle Time has been reached:
[PWXPC_10072] [INFO] [CDCDispatcher] session ended after waiting for
[idle_time] seconds. Idle Time limit is reached
This message is also issued when a continuous extraction is stopped using the PowerExchange
STOPTASK command. In this case, the idle_time variable in the message will show 86400
which is the “never expire” time limit used when an Idle Time of -1 is specified.
Tip: In highly active systems, a positive value for Idle Time may never match. Use 0 if you do
not want the session to run continuously.
For example, if you specify an Idle Time of 10 seconds and PowerExchange finds no data for
the source(s) in the change stream for a 10 second period, PowerExchange will return an EOF
to the Integration Service which will cause it to terminate successfully.
If you specify values for Reader Time Limit and Idle Time, the Integration Service stops
reading data from the source at the point based on whichever one of these session conditions
is reached first. So, if the Reader Time Limit is reached prior to the Idle Time limit, the
session will stop at that point regardless of the fact that Idle Time has not yet been reached.
Warning: Reader Time Limit does not result in normal termination of a CDC session. Use Idle
Time instead of Reader Time Limit.
RestartToken File Folder No Folder name that contains the restart token override file on the Integration
Service node.
Default is $PMRootDir/Restart.
RestartToken File Name No File name in the Restart Token File Folder that contains the restart token
override file. PWXPC uses the contents of this file, if any, in conjunction
with the state information to determine the restart point for the session.
All of these parameters require a valid value and all have valid default values.
There are numerous CDC reader application connection attributes that specify restart
information. PWXPC uses the restart information to tell PowerExchange from which point to
start reading the captured changed data.
Warning: Care must be taken when using the Application Name default as it may not result in
a unique name for the application name. It is imperative that the Application Name value and
the Restart Token File Name be unique for every session. Results are unpredictable and
include session failures and potential data loss if a non-unique name is used for either of these
attributes.
For more information about restart token files, see “Configuring the Restart Token File” on
page 160.
Connection Change
Required Description
Attribute Real-time
UOW Count No Both Specifies the number of units of work (UOWs) you want the
PWXPC to read from the source before flushing data to the target.
If you enter:
-1 = UOW count is not used
0 = UOW count is not used
n = n is the count of UOWs
Default is 1.
A unit of work (UOW) is a collection of changes within a single commit scope made by a
transaction on the source system. Each unit of work may consist of a different number of
changes. This parameter requires a valid value and has a valid default value.
When the session runs, the PWXPC reader begins to read data from the PowerExchange
Listener. After data is provided to the source qualifier, the UOW Count begins. When you
use a non-zero value for the UOW Count session attribute, PWXPC issues a commit to the
target when it reaches the number of units of work specified in this session condition. When
the UOW Count is reached, a real-time flush will be triggered to flush the buffers so that the
changed data is committed to the target. The following message appears in the session log to
indicate that this has occurred:
[PWXPC_10081] [INFO] [CDCDispatcher] raising real-time flush with restart tokens
[restart1_token], [restart2_token] because UOW Count [uow_count] is reached
The commit to the target when reading CDC data is not strictly controlled by the UOW
Count specification. The Real-Time Flush Latency and the Commit Threshold values also
determine the commit frequency. To understand the affect that all of these values have on
commit processing, see “Understanding Commit Processing with PWXPC” on page 121.
For example, if the value for UOW Count is 10, the Integration Service commits all data read
from the source to its target after the 10th unit of work is processed (assuming the Real-Time
Flush Latency period has not expired first). The lower you set the value, the faster the
Integration Service commits data to the target. Therefore, if you require the lowest possible
latency for the apply of changes to the target, you should specify a UOW Count of 1.
Warning: When you specify a low UOW Count value, the session might consume more system
resources on the target platform. This is because it will commit to the target more frequently.
You need to balance performance and resource consumption with latency requirements when
choosing the UOW Count and Real-Time Flush Latency values.
Connection Change
Required Description
Attribute Real-time
Use the Real-time Flush Latency session condition to control the target commit latency when
running in real-time mode. PWXPC commits source data to the target at the end of the
specified maximum latency period. This parameter requires a valid value and has a valid
default value.
When the session runs, PWXPC begins to read data from the source. After data is provided to
the source qualifier, the Real-Time Flush Latency interval begins. At the end of each Real-
Time Flush Latency interval and an end-UOW boundary is reached, PWXPC issues a
commit to the target. The following message appears in the session log to indicate that this
has occurred:
[PWXPC_10082] [INFO] [CDCDispatcher] raising real-time flush with restart tokens
[restart1_token], [restart2_token] because Real-time Flush Latency [RTF_millisecs]
occurred
The CAPI interface timeout also affects latency as it will affect how quickly changes are
returned to the PWXPC reader by PowerExchange. PowerExchange will ensure that it returns
control back to PWXPC at least once every CAPI interface timeout period. This allows the
PWXPC to regain control and, if necessary, perform the real-time flush of data returned. A
high RTF Latency specification will also impact the speed with which stop requests from
PowerCenter are handled as the PWXPC CDC Reader must wait for PowerExchange to
return control before it can handle the stop request.
Tip: Use the PowerExchange STOPTASK command to shutdown more quickly when using a
high RTF Latency value.
For example, if the value for Real-Time Flush Latency is 10 seconds, PWXPC will issue a
commit for all data read after 10 seconds have elapsed and the next end-UOW boundary is
received. The lower you set the value, the faster the data commits data to the target.
Therefore, if you require the lowest possible latency for the apply of changes to the target, you
should specify a low Real-Time Flush Latency value.
Warning: When you specify a low Real-Time Flush Latency interval, the session might
consume more system resources on the source and target platforms. This is because:
♦ The session will commit to the target more frequently therefore consuming more target
resources.
♦ PowerExchange will return more frequently to the PWXPC reader thereby passing fewer
rows on each iteration and consuming more resources on the source PowerExchange
platform.
You need to balance performance and resource consumption with latency requirements when
choosing the UOW Count and Real-Time Flush Latency values.
Connection Change
Required Description
Attribute Real-time
Commit No Real Time Enter the number of change records to process before a commit is
Threshold inserted into the change stream.
Note: This parameter puts a commit record into the change stream after
the specified number of records. The target commit is still controlled by
the UOW Count and Real-time Flush Latency specification.
Default is 0.
Commit Threshold is only applicable to Real-Time CDC sessions. Use the Commit
Threshold session condition to cause commits before reaching the end of the UOW when
processing large UOWs. This parameter requires a valid value and has a valid default value
Commit Threshold can be used to cause a commit before the end of a UOW is received, a
process also referred to as sub-packet commit. The value specified in the Commit Threshold
is the number of records within a source UOW to process before inserting a commit into the
change stream. This attribute is different from the UOW Count attribute in that it is a count
records within a UOW rather than complete UOWs. The Commit Threshold counter is reset
when either the number of records specified or the end of the UOW is reached.
This attribute is useful when there are extremely large UOWs in the change stream that might
cause locking issues on the target database or resource issues on the PowerCenter Integration
Server.
The Commit Threshold count is cumulative across all sources in the group. This means that
sub-packet commits are inserted into the change stream when the count specified is reached
regardless of the number of sources to which the changes actually apply. For example, a
UOW contains 900 changes for one source followed by 100 changes for a second source and
then 500 changes for the first source. If you set the Commit Threshold to 1000, the commit
record is inserted after the 1000th change record which is after the 100 changes for the second
source.
Warning: A UOW may contain changes for multiple source tables. Using Commit Threshold
can cause commits to be generated at points in the change stream where the relationship
between these tables is inconsistent. This may then result in target commit failures.
If 0 or no value is specified, commits will occur on UOW boundaries only. Otherwise, the
value specified is used to insert commit records into the change stream between UOW
boundaries, where applicable.
The value of this attribute overrides the value specified in the PowerExchange DBMOVER
configuration file parameter SUBCOMMIT_THRESHOLD. For more information on this
PowerExchange parameter, see the PowerExchange Reference Manual.
The commit to the target when reading CDC data is not strictly controlled by the Commit
Threshold specification. The commit records inserted into the change stream as a result of the
Commit Threshold value affect the UOW Count counter. The UOW Count and the Real-
Time Flush Latency values determine the target commit frequency.
For example, a UOW contains 1,000 change records (any combination of inserts, updates,
and deletes). If you specify 100 for Commit Threshold and 5 for UOW Count, then a
commit record will be inserted after each 100 records and a target commit will be issued after
every 500 records.
To understand the affect that all of these values have on commit processing, see
“Understanding Commit Processing with PWXPC” on page 121.
Warning: Duplicate data on targets can occur if the Commit On End Of File option in the
Session Properties is enabled. To prevent this, change the Commit Type to Source and then
disable the Commit On End Of File option in the Session Properties. This will ensure that
the only PWXPC issues commits thereby keeping the target data and restart tokens in-sync.
Source-based commit sessions have partitioning restrictions. For more information, see the
PowerCenterWorkflow Administration Guide.
There are three connection attributes which affect commit processing when running real-time
CDC sessions:
1. Commit Threshold
2. Real-Time Flush Latency
3. UOW Count
Note: When using PWXPC CDC Change connections, the only connection attribute that
affects commit processing is UOW Count.
With the exception of Commit Threshold, all source-based commits with PWXPC are done
on end-UOW boundaries. Commit Threshold exists to provide sub-packet commit
capability; that is, to commit data after a specified number of records within a single UOW.
If you specify values for UOW Count and Real-Time Flush Latency, then PWXPC issues
commits to the target when it reaches the UOW Count or when the Real-Time Flush Latency
period expires, whichever occurs first.When the commit is issued both the Real-Time Flush
(RTF) Latency period and the UOW Count counter are reset. PWXPC continues to read data
from the source until either the RTF Latency period matches or the UOW Count matches. At
which point, it issues another commit. This processing continues until the session terminates.
Commits inserted into the change stream as a result of the Commit Threshold specification
are counted in the UOW Count counter. Commit Threshold itself does not result in PWXPC
issuing a commit to the target. Only UOW Count and Real-Time Flush Latency will cause
that to happen.
A final commit is also done by PWXPC during reader termination to ensure that all buffered
and complete UOWs are committed.
Idle Time can impact the commit process by causing the real-time session to terminate. For
example, you set the Idle Time value to 10 seconds and the UOW Count to five units of
work. When the PWXPC reaches the UOW Count, it commits data to the target and
continues to read data from the source. If the Idle Time is then reached, PowerExchange
stops reading from the source and sends an EOF to the Integration Service which terminates
the session. If only 3 complete UOWs have been read since the previous UOW count-based
commit, these will be committed when the session terminates due to the final commit done
by PWXPC.
To illustrate the interaction of all of the values affecting commit processing, assume the
following settings:
♦ Idle Time is -1 (“never timeout”)
♦ Commit Threshold is 0 (no sub-packet commit)
♦ RTF Latency is 5000 (5 seconds)
♦ UOW Count is 1,000
The PWXPC reader receives 900 complete UOWs in five seconds after the first change row
enters the source qualifier. Because the RTF Latency value has matched, a source-based
commit is issued at this point. Both the UOW Count and RTF Latency period are then reset.
So, another commit will not be issued when the 1,000 UOW is read. The next 1,000 UOWs
are read in 4 seconds. So, a commit is issued after these 1,000 UOWs because the UOW
Count matched. Again, the RTF Latency period and the UOW Count are reset at this point.
More changed data is read and commits continue based on whichever attributes matches first.
So, in this example, commits were issued after the first 900 UOWs because RTF Latency
matched first and then again after the 1,900th UOW because the UOW Count then matched
first.
It is therefore possible to have both the Real-Time Flush Latency period and the UOW
Count control commits of the data. Commits will always be done on a UOW boundary based
(except when Commit Threshold is specified) on whichever attribute matches first.
For the lowest latency in getting changed data to the target, use an RTF Latency of 2000 (the
default) and a UOW Count of 1. This will cause a commit at each commit point in the source
data. Since the RTF Latency will only commit on a UOW boundary and a UOW Count of 1
causes a commit after each complete UOW is received, the effect is to have only UOW Count
control the commit process. Of course, this is the most-resource intensive setting from the
target DBMS perspective.
The RTF Latency specification controls:
♦ the commit latency for the target table(s)
♦ the PowerExchange Consumer API (CAPI) interface timeout value (PowerExchange
latency) on the source platform.
In addition to impacting the target latency, high values for RTF Latency will impact the
speed with which stop requests from PowerCenter are handled. The PWXPC CDC Reader
must wait for PowerExchange to return control before it can handle the stop request.
Tip: Use the PowerExchange STOPTASK command to shutdown more quickly when using a
high RTF Latency value.
If you want to use less resources on the target system and want only UOW Count to control
the commit process, then ensure that Real-Time Flush Latency is set sufficiently high so as
not to be a factor. The value necessary to do this is customer-dependent. The size of the
UOWs and the speed at which they are read will affect what value represents a high enough
RTF Latency period to eliminate it as a factor.
Chapter 5
125
PWX_Interfaces_for_PowerCenter.book Page 126 Wednesday, October 3, 2007 9:53 AM
Overview
After you create mappings in the PowerCenter Designer, you can create a session and use the
session in a workflow to extract, transform, and load data. You create sessions and workflows
in the Workflow Manager.
You can create a session in a workflow to extract data in batch, change, or real-time mode.
You determine how you want the Integration Service to extract the data when you configure
the session. You can also create a session to load data to a target.
For more information about creating, configuring, and scheduling workflows, see the
PowerCenter Workflow Administration Guide.
Pipeline Partitioning
Pipeline partitioning cannot be used for sources in CDC sessions. You can use it for targets in
CDC sessions.
For more information about partitioning and a list of all partitioning restrictions, see the
PowerCenter Workflow Administration Guide.
1. In the Task Developer, double-click a session with a non-relational source to open the
session properties.
2. Click the Sources view on the Mapping tab.
3. In the Reader field of the Readers settings, the PowerExchange Batch Reader for the
specific source type will be shown. The Reader names for non-relational batch sources
have one of the following formats:
PowerExchange Batch Reader for <database_type>
4. In the Connection Value field, select the application connection from the available PWX
NRDB Batch connections.
5. Click the Sources view on the Mapping tab.
6. In the Properties settings, configure the following attributes (all fields are optional except
where noted):
Schema Name Override All Overrides the source PowerExchange data map schema name.
Map Name Override All Overrides the source PowerExchange data map name.
File Name ADABAS Unload Specifies the file name of the unloaded Adabas database.
Required for ADABAS Unload.
Database Id Override ADABAS, Overrides the ADABAS Database Id in the PowerExchange data
ADABAS Unload map.
File Id Override ADABAS, Overrides the Adabas file id in the PowerExchange data map.
ADABAS Unload
DB2 Sub System Id DB2 Datamaps Overrides the DB2 instance name in the PowerExchange data
map.
DB2 Table name DB2 Datamaps Overrides the DB2 Table name in PowerExchange data map.
Unload File Name DB2 Unload Overrides the unload file name in PowerExchange data map.
Datasets
or
Use the <group name> syntax to limit the application of the filter
to a specific record.
If you do not specify <group name> and the source mapping is a
multi-record source, then the filter condition applies to all
records in the source mapping.
For example, you can select only records with ID column values
that contain DBA for a multi-record source with two records
called USER1 and USER by specifying either:
USER1=ID=’DBA’;USER2=ID=’DBA’
or
ID=’DBA’
IMS Unload File Name IMS IMS database unload file name. Required if you want to read
source data from the backup file instead of the IMS database.
File Name Override VSAM, SEQ Overrides the data set or file name in the PowerExchange data
map.
Enter enter the complete data set or file name. For the AS400,
this name should be: library_name/file_name.
If you selected the Filelist File check box, enter the filelist file
name in the File Name Override.
Filelist File VSAM, SEQ Select if the File Name Override field contains the data set name
of a list of files. Only select this option if you have entered a
filelist file for File Name Override.
SQL Query Override All Overrides the SQL query sent to PowerExchange, including any
Filter Overrides.
For information about other session properties, see the PowerCenter Workflow
Administration Guide.
7. Click OK.
1. In the Task Developer, double-click a session with a relational source to open the session
properties.
2. Click the Sources view on the Mapping tab.
3. In the Reader field of the Readers settings, select Relational Reader. For DB2 on z/OS,
you can also select PowerExchange Reader for DB2 Image Copy.
For example, the available readers for a DB2 source looks as follows:
Note: The PowerExchange Reader for DB2 Image Copy is only applicable to DB2 for
z/OS sources. It cannot be used for DB2 sources on other platforms.
4. In the Connection Value field, select PWX NRDB Batch application connection if using
the DB2 Image Copy reader.
Select the appropriate PWX relational database connection if using the relational reader:
♦ PWX DB2390 for DB2 for z/OS
♦ PWX DB2400 for DB2 for i5/OS
♦ PWX DB2UDB for DB2 for Linux, UNIX, and Windows
♦ PWX Oracle for Oracle
♦ PWX MSSQLServer for SQL Server
♦ PWX Sybase for Sybase
5. You can configure the following attributes for the PowerExchange Reader for DB2 Image
Copy:
DB2 Sub System Id Overrides the DB2 instance name in the PowerExchange data map.
Image Copy Dataset Provides the image copy data set name. If not specified, the most current
image copy data set is used.
Disable Consistency Checking Overrides the unload file name in PowerExchange data map.
Filter Overrides Allows data to be filtered at the source by PowerExchange. Enter a filter
override using the following syntax:
For example, you can select only records with ID column values that contain
DBA by specifying:
ID=’DBA’
SQL Query Override Overrides the SQL query sent to PowerExchange, including any filter
overrides.
For information about other session properties, see the PowerCenter Workflow
Administration Guide.
6. Click OK.
If you want to extract change data from a multi-record non-relational source using extraction
maps, you must create a PowerExchange capture registration for every table in the data map.
This creates an extraction map for each table. You can then either import the data map as a
multi-record non-relational source (for batch usage) or import the extraction maps for each
table (for CDC usage).
The Custom Property can be set in the session Config Object tab in the Custom Properties
attribute. This property can also be set in the Integration Service making it applicable to all
workflows and sessions that use that Integration Service. See Knowledge Base Article #18015
for information about how to set Custom Properties in the Integration Service.
If you are using full constraint-based loading, then your mapping must not contain active
transformations which change the Row Id generated by the CDC Readers. The
transformations that change the Row Id are: Aggregator, Custom (configured as active),
Joiner, Normalizer, Rank, and Sorter Transformations. All other transformations can be used.
For additional real-time mode limitations, see the PowerCenter Workflow Administration
Guide.
For more information about constraint-based loading, see the PowerCenter Workflow
Administration Guide.
To enable recovery for the session, change the Recovery Strategy attribute to Resume from
last checkpoint. Enabling recovery for CDC sessions is important to ensure that data and
restart tokens are properly handled. For more information, see “PWXPC - Restart and
Recovery” on page 147.
3. Click the Sources view on the Mapping tab.
For relational sources, you will have to choose the desired CDC Reader (see page 137).
With extraction map sources, the reader is automatically chosen based on the source type
of the extraction map. In this example, the extraction map used in the mapping is for a
DB2/390 source so the PowerExchange CDC Reader for DB2/390 is chosen:
In the Connection Value field, select CDC Real Time or CDC Change application
connection types. PWXPC displays the valid connections for the source type in the
Application Connection Browser.
4. Optionally, open the application connection to override any connection values. See
“PWXPC - Configuring Connections” on page 77.
5. In the Session Properties settings, configure the following optional attributes:
Schema Name All Overrides the source PowerExchange extraction map schema
Override name.
Map Name Override All Overrides the source PowerExchange extraction map name.
Database Id Override ADABAS Overrides the ADABAS Database Id in the PowerExchange data
map.
File Id Override ADABAS Overrides the Adabas file id in the PowerExchange data map.
Library/File Override DB2400 Overrides the library and file name in the extraction map.
Specify complete library and file names in the format:
library/file
SQL Query Override Overrides the SQL query sent to PowerExchange, including any
Filter Overrides.
For information about other properties settings, see the PowerCenter Workflow
Administration Guide.
In the Reader field of the Readers settings, select a CDC Reader from those available
based on the source type:
♦ DB2390: PowerExchange CDC Change or PowerExchange CDC Real Time
♦ DB2400: PowerExchange CDC Change or PowerExchange CDC Real Time
♦ DB2UDB: PowerExchange Real Time
♦ Oracle: PowerExchange CDC Change or PowerExchange CDC Real Time
♦ MSSQL: PowerExchange Real Time
4. In the Connection Value field, select an application connection.
Tip: For CDC Change and Real Time Readers, only choose an application connection for
the first CDC source. For subsequent CDC sources, choose a Connection Type of None.
PowerExchange group source processing only uses the information on the first
application connection. Subsequent application connection specifications are not
required and may cause session failures.
5. Optionally, open the application connection to override any connection values. See
“PWXPC - Configuring Connections” on page 77.
6. In the Properties settings, configure the following attributes. All fields are optional except
where noted:
Extraction Map Name All The PowerExchange extraction map name for the CDC source.
Required if using a CDC Change or Real Time Reader.
You must specify the extraction map name for the relational
source.
Library/File Override DB2400 Real Time Overrides the library and file name in the extraction map.
Specify complete library and file names in the format:
library/file
Source Schema Oracle Change and Overrides the source schema name.
Override Real Time
For information about other properties settings, see the PowerCenter Workflow
Administration Guide.
7. Click OK.
4. In the Connections Value field, select a relational database connection from one of the
following types:
♦ PWX DB2390 for DB2 for z/OS
♦ PWX DB2400 for DB2 for i5/OS
♦ PWX DB2UDB for DB2 for Linux, UNIX, and Windows
♦ PWX Oracle for Oracle
♦ PWX MSSQLServer for SQL Server
♦ PWX Sybase for Sybase
5. Configure any other session properties.
6. Click OK.
For information about properties settings, see the PowerCenter Workflow Administration
Guide.
3. In the Connections Value field, select a PWX NRDB Batch application connection.
4. In the Properties settings, configure the PWXPC session properties. See Table 5-1 on
page 144.
5. Configure any other session properties.
6. Click OK.
For information about other Properties settings, see the PowerCenter Workflow
Administration Guide.
Delete SQL Override ADABAS, IMS, VSAM Overrides default Delete SQL sent to PowerExchange.
File Name Override SEQ, VSAM Overrides the data set or file name in the PowerExchange data
map.
Enter enter the complete data set or file name. For the AS400, this
name should be: library_name/file_name.
Insert Only ADABAS, IMS, VSAM Processes updates and deletes as inserts.
Insert SQL Override All Overrides the default Insert SQL sent to PowerExchange.
LRECL SEQ (MVS only) MVS data set logical record length.
Default is 256.
Map Name Override All Overrides the target PowerExchange data map name.
MODELDSCB SEQ (MVS only) MVS Model DSCB for non-SMS-managed GDG data sets.
Post SQL All SQL statement(s) executed after the session is run using the target
database connection.
Pre SQL All SQL statement(s) executed before the session is run using the
target database connection.
Primary Space SEQ (MVS only) MVS Primary space allocation, in the units specified in Space.
Default is 1.
RECFM SEQ (MVS only) MVS Record format. Values are: F, V, FU, FB, VU, VB, FBA, VBA.
Schema Name All Overrides the target PowerExchange data map schema name.
Override
Secondary Space SEQ (MVS only) MVS Secondary space allocations, in the units specified in Space.
Default is 1.
Space SEQ (MVS only) MVS data set space units. Values are:
- CYLINDER
- TRACK
Default is TRACK.
Truncate Table IMS, VSAM Truncates table (deletes contents) before loading new data.
Option Note: VSAM data sets must be defined with the REUSE option for
truncate to function correctly.
Update SQL Override ADABAS, IMS, VSAM Overrides the default Update SQL that is sent to PowerExchange.
Upsert ADABAS, IMS, VSAM Processes failed inserts as updates and updates as inserts.
Chapter 6
147
PWX_Interfaces_for_PowerCenter.book Page 148 Wednesday, October 3, 2007 9:53 AM
Overview
This chapter describes PWXPC restart and recovery processing as well as how to configure
your CDC sessions to use this processing.
Each source in a CDC session has unique restart information, also referred to as restart
tokens. PWXPC manages the CDC restart information. The Integration Service provides
recovery for the target files and tables in CDC sessions.
In order to extract change data from the change stream, PWXPC provides restart information
for the CDC sources to PowerExchange. PowerExchange reads the change stream on the
CDC source platform and provides complete units of work to PWXPC. A unit of work
(UOW) is a collection of changes within a single commit scope made by a transaction on the
source system. Using the commit interval information specified in the CDC session
connection, PWXPC periodically flushes complete UOWs to the Integration Service.
Target recovery and restart information is stored as the target tables and files are updated by
the Integration Service. The Integration Service and PWXPC use this information to recover
and restart stopped or failed sessions from the point of interruption.
In order to properly configure CDC sessions, review the following topics:
♦ “CDC-Specific Connection Attributes” on page 110
♦ “Understanding Commit Processing with PWXPC” on page 121
♦ “Extracting CDC Data in Change and Real-time Modes” on page 133
♦ “Understanding PWXPC Restart and Recovery” on page 149
♦ “Configuring the Restart Token File” on page 160
♦ “Enabling Session Recovery” on page 163
♦ “Configuring CDC Sessions” on page 164
In order to manage CDC sessions, review “PWXPC Restart and Recovery Operation” on
page 163.
Session Recovery
PowerCenter and PWXPC restart and recovery functionality are utilized when you configure
the Recovery Strategy Resume from the last checkpoint. Enabling a resume recovery strategy
ensures that both the target data and CDC restart information can be recovered in the event
of session failure.
Warning: Do not enable recovery processing if any of the targets in the CDC session use the
File Writer to write CDC data to flat files. The restart tokens for all targets in the session,
including relational targets, will be compromised if there is a flat file target in the same
session. Data loss or duplication may occur.
When you enable a resume recovery strategy, the Integration Service provides recovery for the
target tables and files and PWXPC provides recovery for the CDC restart information.
PWXPC issues the following message indicating that recovery is in effect:
PWXPC_12094 [INFO] [CDCRestart] Advanced GMD recovery in affect. Recovery is automatic
The Integration Service stores the session state of operation in the shared location,
$PMStorageDir. The Integration Service saves relational target recovery in the target
database.
CDC restart information, also called restart tokens, originates from PowerExchange on the
CDC source platform. PWXPC stores CDC restart information in different locations based
upon the target type:
♦ For non-relational targets, PWXPC stores the CDC restart information in the shared
location, $PMStorageDir, in state files on the Integration Service platform.
♦ For relational targets, PWXPC stores the CDC restart information in state tables in the
target database.
When the Integration Service performs recovery, it restores the state of operation to recover
the session from the point of interruption. It uses the target recovery data to determine how
to recover the target tables. PWXPC and PowerExchange use the CDC restart information to
determine the correct point in the change stream from which to restart the extraction.
Recovery Tables
For relational targets, the Integration Service creates the following recovery tables in the
target database:
♦ PM_RECOVERY. This table contains target load information for the session run. The
Integration Service removes the information from this table after each successful session
and initializes the information at the beginning of subsequent sessions.
♦ PM_TGT_RUN_ID. This table contains information the Integration Service uses to
identify each target on the database. The information remains in the table between session
runs. If you manually create this table, you must create a row and enter a value other than
zero for LAST_TGT_RUN_ID to ensure that the session recovers successfully.
♦ PM_REC_STATE. This table contains restart information for CDC sessions. The restart
information recorded in the table contains the application name and restart tokens for the
session. The restart information remains in the table permanently. The Integration Service
updates it with each commit to the target tables.
If you edit or drop the recovery tables before you recover a session, the Integration Service
cannot recover the session. If you disable recovery, the Integration Service does not remove
the recovery tables from the target database. You must manually remove the recovery tables.
If you want the Integration Service to create the recovery tables, grant table creation privilege
to the database user name for the target database connection. For the database user name used
with PowerExchange relational targets, see “Recovery Table Creation with PowerExchange
Targets” on page 158.
If you do not want the Integration Service to create the recovery tables, create the recovery
tables manually.
Tip: If you are using PowerExchange relational target connections, manually create these
tables so you can assign the desired database attributes. See “Creating the Recovery Tables
Manually” on page 159.
For more information about the PM_RECOVERY and PM_TGT_RUN_ID tables, see the
PowerCenterWorkflow Administration Guide.
The Integration Service uses the application name from the source CDC connection for the
application name value in the state file name prefix. The Integration Service includes the
complete file name in message CMN_65003. The remainder of the fields in the file name are
task and workflow repository attributes. These repository attributes remain static unless the
task or workflow is altered. The following examples are actions that alter these repository
attributes:
♦ adding or removing sources or targets from the mapping used by the session
♦ moving the workflow or session to a different folder
♦ moving the session to a different workflow
See “Changing CDC Sessions” on page 169 for additional information.
<restart_token_file_name>yyyymmddhhmmss_term
where:
♦ restart_token_file_name is the restart token file name from the CDC connection
♦ yyyymmddhhmmss is the initialization file creation timestamp
♦ init or term is for initialization and termination files, respectively
For example, a CDC source application connection specifies a restart token file name of
my.app.txt, which does not exist. PWXPC creates the following files on the Integration
Service platform in the restart token file folder specified in the connection:
♦ my.app.txt
♦ my.app.txtyyyymmddhhmmss_init
♦ my.app.txtyyyymmddhhmmss_term
The restart token file, my.app.txt, is empty. The timestamps on both the initialization and
termination files are the same to indicate that they are related to the same run. The
termination file may not exist or may be empty if the session fails.
If you are using the default value of zero for the connection attribute Number of Runs to Keep
RestartToken File, PWXPC keeps only one copy of the paired initialization and termination
files. Otherwise, PWXPC uses the value specified in that attribute to determine the number
of backup copies of these paired files to keep. During termination, PWXPC removes any
additional pairs of the backup files beyond the Number of Runs to Keep RestartToken File value.
For example, a target table has been materialized from its source data and no new changes
have been made to the source data. Now you need to establish a starting extraction, or restart,
point in the change stream. You do this by using DTLUAPPL. DTLUAPPL is a
PowerExchange utility that generates restart points. After you have run DTLUAPPL, place
the generated restart tokens in the restart token file specified in the source CDC connection
and cold start the CDC session. PWXPC passes the restart tokens from the restart token file
to PowerExchange. PowerExchange extracts changes from the change stream from that restart
point forward.
Table 6-1 describes earliest starting extraction (restart) points PowerExchange uses if null
restart tokens are supplied for all sources:
Source
Platform/ CDC Change Connection CDC Real Time Connection
Database
MVS (all Oldest Condense file recorded in the Logger selects the best available restart point.
sources) CDCT. This is the oldest restart point for which an archive log
is available, or active log if there are no available
archive logs.
AS400 Oldest Condense file recorded in the Oldest journal receiver still on the journal receiver
CDCT. chain.
Oracle Oldest Condense file recorded in the Most current Oracle catalog dump.
CDCT.
UDB n/a Current log position at the time the capture catalog
was created.
PowerExchange only uses the default starting extraction point if all sources have null restart
tokens. PWXPC assigned the oldest restart point of the restart tokens available if there are
some sources without restart tokens.
For example, a new CDC session contains three sources called A, B, and C. The restart token
file contains restart points for sources A and B. The restart point for source A is older than
source B. Source C has no existing or supplied restart point. When you run the session,
PWXPC assigns source C the same restart point as source A since it is the oldest supplied
restart point. PWXPC does not assign the default starting extraction point discussed in Table
6-1 to source C because some sources have restart points.
♦ Creates the initialization restart token file containing the reconciled restart information
PWXPC passes the restart tokens for all sources to PowerExchange. PowerExchange uses the
oldest restart token passed by PWXPC to start extracting data from the change stream.
PowerExchange does not pass data for a source until its restart point is reached. This prevents
targets from being updated with records processed in previous extraction runs.
PWXPC continually updates the restart tokens for each source in the state table or the state
file as it issues flushes target data. With relational target tables in the same database, the
Integration Service updates both the target tables and the restart tokens within a single
commit. The Integration Service does separate commits for each unique relational database.
With heterogeneous target, the restart tokens in one relational database may differ from those
in another relational database at specific points in time.
When using non-relational targets, the state file and the targets likely exist on completely
different machines. With non-relational targets, the Integration Service updates the targets
and the state file in separate operations. If the session fails after the Integration Service
commits data to the target but before it updates the restart tokens in the state file, targets may
receive duplicate data when restarted. On warm start, PWXPC uses the last restart tokens
written prior to the failure. As a result, PWXPC re-sends data which has already been applied
to the non-relational targets.
The Integration Service commits the flushed data to the targets, including the restart tokens
for relational targets. After the Integration Service writes the flushed data to any non-
relational targets, it updates the state file with the restart tokens.
If the session fails, the Integration Service rolls back any uncommitted data and the related
restart tokens for relational targets. This leaves only the last successfully committed UOW
data and restart tokens in the relational target tables. The Integration Service uses relational
database rollback capabilities to ensure that uncommitted data is removed during session
termination. Consistency between the restart tokens and the relational target data is
guaranteed because they are both committed within the same commit scope.
The Integration Service does not do rollback processing for non-relational targets. As a result,
duplicate data can occur on restart. You should account for this in your CDC session design.
Tip: If the possibility of duplicate data is unacceptable to your application, then design your
CDC sessions to use only relational targets.
Run one of the following scripts to create the recovery tables in the target database:
Script Database
create_schema_db2.sql DB2
create_schema_inf.sql Informix
create_schema_ora.sql Oracle
create_schema_syb.sql Sybase
create_schema_ter.sql Teradata
This is generic DDL. Make the appropriate changes for your environment.
Syntax Rules
The restart token file has these syntax rules:
♦ No blank lines between statements.
♦ Comment lines begin with:
<!--
RESTART1=restart1_token
RESTART2=restart2_token
extraction_map_name=restart1_token
extraction_map_name=restart2_token
...
extraction_map_namen=restart1_token
extraction_map_namen=restart2_token
Comment Statement
<!-- comment text
Use the comment statement anywhere in the restart token file. The “<!--” is required.
The explicit override statement specifies restart tokens for a specific source. The source is
defined by specifying the extraction map name. Sources can have multiple extraction
mappings and therefore multiple extraction map names. Each source specification must be a
pair of lines with:
♦ The source extraction map name (extraction_map_name) specified with the restart1_token
value
♦ The source extraction map name (extraction_map_name) specified with the restart2_token
The extraction map name specified in the restart token file must match what is defined in the
CDC session. To determine the extraction map name:
♦ Check the Extraction Map Name attribute in the Session Properties for relational sources
♦ Check the Schema Name Override and Map Name Override attributes in the Session
Properties if using CDC datamap sources. See Figure 5-4 on page 136. These attributes
override the source name in the CDC datamap source.
♦ Check the Schema Name and Map Name values in the source Metadata Extensions in
Designer if using CDC datamap sources. See Figure 3-23 on page 58.
The restart1_token value varies based on capture source and is found in the following:
♦ Sequence= value, minus the trailing 8 zeros, in the DTLUAPPL PRINT output
♦ DTL__CAPXRESTART1 value when extracting data
♦ Sequence token in PowerExchange messages (e.g., PWX-04564, PWX-09959)
♦ Restart Token 1 in PWXPC messages (e.g. PWXPC_12060, PWXPC_12069)
The restart2_token value varies based on the capture source and is found in the following:
♦ Restart= value in the DTLUAPPL PRINT output
♦ DTL__CAPXRESTART2 value when extracting data
♦ Logger token in PowerExchange messages (e.g., PWX-04564, PWX-09959)
♦ Restart Token 2 in PWXPC messages (e.g. PWXPC_12060, PWXPC_12069)
If the session includes source extraction maps that do not have entries in the existing restart
token file, then the session executes without error. See “Default Restart Points” on page 155
for an explanation of what restart tokens PWXPC and PowerExchange use in this case.
The special override statement allows specifies for all sources in a session. The restart token
values (restart1_token and restart2_token) are described in detail in the Explicit Override
Statement.
If used, both RESTART1= and RESTART2= must be specified.
This override can be used in conjunction with explicit override statements to provide restart
tokens for sources which do not have explicit override statements.
An explicit override statement for a source takes precedence over the special override
statement for that source.
In the example, the session contains 7 source tables. The restart token file contains explicit
override statements for 3 sources: RRTB_SRC_001, RRTB_SRC_002, and RRTB_SRC_004.
It also contains the special override statement to provide the restart tokens for the remainder
of the sources in the session. When the session executes, PWXPC issues message
PWXPC_12060 as follows:
===============================
Session restart information:
===============================
Extraction Map Name Restart Token 1 Restart Token 2 Source
d1dsn7.rrtb0001_RRTB_SRC_001 0000060D1DB2000000000000060D1DB20000000000000000 C1E4E2D340400000013FF36200000000 Restart file
d1dsn7.rrtb0002_RRTB_SRC_002 000000A3719500000000000000A371950000000000000000 C1E4E2D34040000000968FC600000000 Restart file
d1dsn7.rrtb0003_RRTB_SRC_003 000000AD775600000000000000AD77560000000000000000 C1E4E2D34040000000AD5F2C00000000 Restart file (special override)
d1dsn7.rrtb0004_RRTB_SRC_004 000006D84E7800000000000006D84E780000000000000000 C1E4E2D340400000060D1E6100000000 Restart file
d1dsn7.rrtb0005_RRTB_SRC_005 000000AD775600000000000000AD77560000000000000000 C1E4E2D34040000000AD5F2C00000000 Restart file (special override)
d1dsn7.rrtb0006_RRTB_SRC_006 000000AD775600000000000000AD77560000000000000000 C1E4E2D34040000000AD5F2C00000000 Restart file (special override)
d1dsn7.rrtb0007_RRTB_SRC_007 000000AD775600000000000000AD77560000000000000000 C1E4E2D34040000000AD5F2C00000000 Restart file (special override)
PWXPC displays the sources with explicit overrides with “Restart file” under the Source
column. The sources to which PWXPC assigns the special override restart tokens have
“special override” in parentheses.
PWXPC automatically recovers warm started sessions when a resume recovery strategy is
specified.
Application Names
PowerExchange, when using ODBC connections, stores the restart tokens in the
PowerExchange CDEP file on the extraction platform. PWXPC stores the restart tokens in:
♦ The state file on the Integration Service platform, for non-relational sources
♦ The state table in the target database, for relational sources
PowerExchange always stores extraction history information in the CDEP file for each
application name, regardless of whether the CDEP is being used to maintain restart tokens or
not. With PWXPC the CDEP file is used for history only.
An application name is required when using PWXPC. Each CDC session must use a unique
application name in order to prevent failures due to conflicts in the CDEP. Application names
cannot be shared with other CDC sessions.
Warning: Do not use a PWXPC CDC session application name when performing a Database
Row Test in PowerExchange Navigator or when using the DTLUAPPL utility. Using the same
application name as a PWXPC CDC session in another CDC session, a Navigator row test, or
in DTLUAPPL fails with message:
PWX-04553 Error restart tokens [required | not allowed] for application
"application name"
Use a unique application name when generating restart tokens with DTLUAPPL so you avoid
any conflicts with existing application names used for CDC sessions.
For more information on DTLUAPPL, see the PowerExchange Utilities Guide.
For more information on configuring the restart token file, see “Configuring the Restart
Token File” on page 160.
The DTLUAPPL prints the generated tokens because the PRINT APPL statement is
specified:
Application name=<tokens> Rsttkn=<1> Ainseq=<0> Preconfig=<N>
FirstTkn =<>
LastTkn =<>
CurrentTkn=<>
Registration name=<db2demo1.1> tag=<DB2DSN7db2demo11>
Sequence=<000007248B9600000000000007248B9600000000>
Restart =<D2D1D4D34040000007248B0E00000000>
DTLUAPPL does not generate the complete restart1_token value which is shown in the
SEQUENCE token. You must be add the trailing four bytes (eight digits) of zeros manually
when you update the restart token file. DTLUAPPL does generate the complete
restart_token2 value in the Restart= token. You can copy this value to the restart token file.
Tip: You can use same restart tokens for multiple source tables in the restart token file to start
extracting changes from the same point in the change stream. You only need to run
DTLUAPPL multiple times if you want to start extracting changes from different locations in
the change stream for different sources.
Using the tokens in this example, the restart token file looks as follows:
D1DSN7.db2demo1=000007248B9600000000000007248B960000000000000000
D1DSN7.db2demo1=D2D1D4D34040000007248B0E00000000
PWXPC automatically performs recovery when a workflow or task is warm started. You do
not need to recover workflows and tasks before you restart them.
PWXPC performs the following tasks for warm start:
1. PWXPC reconciles the restart tokens from the restart token file and from the recovery
state tables and file. See “Determining the Restart Point” on page 153.
2. For heterogeneous targets, PWXPC queries the Integration Service about the commit
levels of all targets. If all targets in the session are at the same commit level, PWXPC
skips recovery processing.
3. If recovery is required for heterogeneous targets, PWXPC re-reads the data for the last
UOW committed to higher-level targets and flushes it to those targets with the lower
commit level. The Integration Service commits any flushed data and restart tokens to any
relational targets and updates any non-relational files.
Note: PWXPC does not read the restart token file if recovery is required.
4. If recovery is not required and the reconciled restart tokens differ from those in the state
tables and file, PWXPC commits the reconciled restart tokens and issues message
PWXPC_12104.
5. PWXPC continues processing and committing data and restart tokens until the session
ends or is stopped.
Recovery Processing
Recover workflows and tasks by selecting the recover command in Workflow Manager,
Workflow Monitor, or pmcmd. When you request recovery, PWXPC issues the following
message:
PWXPC_12093 [INFO] [CDCRestart] Recovery run requested. Targets will be
resynchronized if required and processing will terminate
Select Recover to populate the restart token file with the restart tokens for all sources in the
CDC session so that you can cold start. You can also use recovery to ensure the targets and
restart tokens are in a consistent state.
PWXPC automatically performs recovery when a workflow or task is warm started. You do
not need to recover workflows and tasks before you restart them. PWXPC performs the
following tasks for recovery:
1. PWXPC reads the restart tokens from the recovery state tables and file.
2. PWXPC creates the initialization restart token file with the reconciled restart tokens.
3. For heterogeneous targets, PWXPC queries the Integration Service about the commit
levels of all targets. If all targets in the session are at the same commit level, PWXPC
skips recovery processing.
4. If recovery is required for heterogeneous targets, PWXPC re-reads the data for the last
UOW committed to higher-level targets and flushes it to those targets with the lower
commit level. The Integration Service commits any flushed data and restart tokens to any
relational targets and updates any non-relational files.
Note: PWXPC does not read the restart token file if recovery is required.
5. PWXPC updates the restart token file with the final restart tokens, creates the
termination restart token file, and ends.
You can now warm start or cold start the workflow or task to process changed data from the
point of interruption.
When you stop a workflow or task gracefully by issuing the stop command through
PowerCenter or PowerExchange, the following action occurs:
1. The Integration Service requests PWXPC to stop if you issue the PowerCenter stop
command.
If you issue the PowerExchange stop command, it sends an end of file to PWXPC.
2. PWXPC performs end of file processing to flush the remaining uncommitted complete
units of work to the targets and issues message PWXPC_12101. PWXPC also commits
the restart tokens and issues message PWXPC_12068.
3. The Integration Service processes all of data in the pipeline and writes it to the targets.
4. The Integration Service sends an acknowledgement to PWXPC indicating that the
targets have been updated.
5. PWXPC issues message PWXPC_12075, writes the termination restart token file, and
shuts down.
6. The Integration Service ends the session successfully.
You can also stop CDC sessions gracefully if you:
♦ Use Idle Time=0 in the PWX CDC Real Time connection which instructs PowerExchange
to stop processing at end of log. See “Configuring Idle Time” on page 113.
♦ Use a PWX CDC Change connection to extract changes from condense files.
When you use PowerExchange batch change extraction mode for condense files, the
extraction automatically ends when all condensed data is read.
You can also stop a workflow or task using the abort command in Workflow Monitor or
pmcmd. For information about the abort command, see the PowerCenterWorkflow
Administration Guide.
To change a CDC session and then restart it from the point of interruption, perform the
following tasks:
1. Gracefully stop the workflow. See “Stopping CDC Sessions” on page 168.
2. After the workflow stops successfully, issue the Recover command for the CDC session.
When you recover tasks, PWXPC writes the ending restart tokens for the session into the
restart token file.
3. Change the session or workflow as desired.
4. Ensure that the restart token file specified in the source CDC connection specifies the
restart token file updated in the recovery session.
5. Optionally, update the restart token file to add or remove sources.
6. Cold start the CDC session.
===============================
Session restart information:
===============================
Extraction Map Name Restart Token 1 Restart Token 2 Source
d1dsn7.rrtb0002_RRTB_SRC_002 000000AD220F00000000000000AD220F0000000000000000 C1E4E2D34040000000AD0D9C00000000 GMD storage
d1dsn7.rrtb0001_RRTB_SRC_001 000000AD220F00000000000000AD220F0000000000000000 C1E4E2D34040000000AD0D9C00000000 GMD storage
d1dsn7.rrtb0003_RRTB_SRC_003 000000AD220F00000000000000AD220F0000000000000000 C1E4E2D34040000000AD0D9C00000000 GMD storage
PWXPC places the restart tokens in the restart token file specified in the CDC
application connection.
3. Make any necessary changes to the mapping, session, and workflow to add the new
source, RRTB_SRC_004.
4. Run DTLUAPPL with RSTTKN GENERATE to generate restart tokens for the current
end-of-log. Use the following DTLUAPPL control cards to do this:
mod APPL dummy DSN7 rsttkn generate
mod rsttkn rrtb004
end appl dummy
print appl dummy
Add eight zeroes to the end of the Sequence= value to create the restart token file value.
5. Update the restart token file to add the new source and its tokens. The updated file looks
as follows:
<!-- existing sources
d1dsn7.rrtb0001_RRTB_SRC_001=000000AD220F00000000000000AD220F0000000000000000
d1dsn7.rrtb0001_RRTB_SRC_001=C1E4E2D34040000000AD0D9C00000000
d1dsn7.rrtb0002_RRTB_SRC_002=000000AD220F00000000000000AD220F0000000000000000
d1dsn7.rrtb0002_RRTB_SRC_002=C1E4E2D34040000000AD0D9C00000000
d1dsn7.rrtb0003_RRTB_SRC_003=000000AD220F00000000000000AD220F0000000000000000
d1dsn7.rrtb0003_RRTB_SRC_003=C1E4E2D34040000000AD0D9C00000000
<!-- new source
d1dsn7.rrtb0004_RRTB_SRC_004=00000DBF240A0000000000000DBF240A0000000000000000
d1dsn7.rrtb0004_RRTB_SRC_004=C1E4E2D3404000000DBF238200000000
PWXPC automatically recovers when the session is warm started. PWXPC issues the
following messages displaying the restart tokens found for the session and its sources:
CDCDispatcher> PWXPC_12060 [INFO] [CDCRestart]
===============================
Session restart information:
===============================
Extraction Map Name Restart Token 1 Restart Token 2 Source
d1dsn8.rrtb0004_RRTB_SRC_004 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
d1dsn8.rrtb0009_RRTB_SRC_009 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
d1dsn8.rrtb0005_RRTB_SRC_005 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
d1dsn8.rrtb0006_RRTB_SRC_006 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
d1dsn8.rrtb0008_RRTB_SRC_008 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
d1dsn8.rrtb0003_RRTB_SRC_003 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
d1dsn8.rrtb0002_RRTB_SRC_002 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
d1dsn8.rrtb0001_RRTB_SRC_001 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
d1dsn8.rrtb0007_RRTB_SRC_007 00000FCA65840000000000000D2E004A00000000FFFFFFFF C1E4E2D3404000000D21B1A500000000 GMD storage
PWXPC issues the PWXPC_12069 message when it detects that recovery is required. The
12069 message usually includes the begin-UOW (from) and end-UOW (to) restart tokens for
the oldest uncommitted UOW that PWXPC re-reads during recovery. PWXPC stores end-
UOW restart tokens in the state table and file unless sub-packet commit is used. See
“Configuring Commit Threshold” on page 119.
CDCDispatcher> PWXPC_12069 [INFO] [CDCRestart] Running in recovery mode. Reader will resend the the oldest uncommitted UOW to
resync targets:
from: Restart 1 [00000FCA65840000000000000D2E004A00000000FFFFFFFF] : Restart 2 [C1E4E2D3404000000D21B1A500000000]
to: Restart 1 [00000FCA65840000000000000D300D8000000000FFFFFFFF] : Restart 2 [C1E4E2D3404000000D21B1A500000000].
The from restart tokens are the same as those displayed in the PWXPC_12060 messages for all
sources. This restart token represents the start point in the change stream for the oldest
uncommitted UOW. The to restart tokens represent the end of the oldest uncommitted
UOW. Since the application connection for this session specifies sub-packet commit, the
Restart 2 value is the begin-UOW value in both cases. The Restart 1 values represent the start
and end change records in the Restart 2 UOW.
PWXPC rereads the changes between the two restart token values in the 12069 message and
issues a commit for the data and the restart tokens. The Integration Service writes the data to
the target tables and the restart tokens to the state table.
PWXPC and the Integration Service continue to read and write data and restart tokens until
the session ends or is stopped.
Save session log by attribute to keep session logs by timestamp instead of number of runs. For
more information about archiving session logs, see the PowerCenterWorkflow Administration
Guide.
PWXPC keeps a single pair of initialization and termination restart token files. In the source
CDC connection, request more than a single copy by specifying a value larger than 1 in the
Number of Runs to Keep RestartToken File attribute.
Figure 6-1 shows an example of an Application Connection that specifies that 10 copies of the
restart token file be maintained in the Restart Token File Folder:
You can determine starting and ending restart points for each extraction using historical
copies of the restart token file. You need historical copies of the session logs to re-extract
changes at any point in between the session start and end.
When PWXPC issues a real-time flush to commit data to the targets, it issues message
PWXPC_10081. This message contains the restart tokens at that point in time:
PWXPC_10081 [INFO] [CDCDispatcher] raising real-time flush with restart tokens
[<restart1_token>], [<restart2_token>] <because UOW Count [<n>] is reached.> | <because
Real-time Flush Latency [<n>] occurred.>
To restart an extraction from a specific commit point, use the restart tokens in the
appropriate 10081 message to populate the restart token file and cold start the CDC session.
PWXPC passes the restart token file values to PowerExchange to extract the data from that
point forward.
Chapter 7
175
PWX_Interfaces_for_PowerCenter.book Page 176 Wednesday, October 3, 2007 9:53 AM
Here we see the that COL1 is updated from an A to a C. COL1 is marked as the primary key
for the target table and is therefore treated as a delete of row A and an insert of row C.
1. To add the before image and change indicator right click on the required extraction
group.
2. Open the group as shown below:
3. Open the required extraction map by either right-clicking and selecting open , or by
double-clicking on the required extraction map.
4. Right-click on the column which requires the before image and change indicator to be
set.
6. Select and add (by double-clicking or highlighting and then using the “Add >>” button)
the required column to set the change indicator. When all the required change indicators
have been set up, select the Before Images tab.
7. Repeat the process for columns which will require the before image to be included with
the change capture data.
Note: The BI and CI column names can be changed on these screens by single-clicking and
editing. The following sections in this chapter will refer to the default names.
Configuring PowerCenter
The PowerExchange Change Data Capture (CDC) source will now need to be imported. This
must be done through the Import from PowerExchange option under Source Analyzer in the
PowerCenter Designer. For more information on importing a CDC source see “Working with
Extraction Map Definitions” on page 53.
After the CDC source has been imported, the target definitions will be required. Import them
using the Target Designer in the PowerCenter Designer. For more information about
importing a target see “Source and Target Definitions in Designer” on page 19.
The following example will work with a DB2 data capture source and a DB2 target table. The
structure of the source CDC table is:
Note: The columns prefixed DTL_CI and DTL_BI are the change indicator and before image
respectively which are vital for this flexible transformation. The DTL__CAPX columns are
PowerExchange capture columns.
The target is a DB2 table of the following structure:
The source table has a primary key of CUSTOMER_ID, and the PARTNER_CUST_ID is
an attribute of the table. The transformation will load the data into the target table where the
PARTNER_CUST_ID is the primary key, and the CUSTOMER_ID is an attribute of that
table.
In this case, if the PARTNER_CUST_ID changes in the source table, a delete and insert will
be required for the target.
Now drag the PowerExchange CDC source, and the target into the mapping as below:
Note: Any of the control information including BI and CI columns could, if required, be
defined in the target definition.
Note: The example above shows a single column primary key. Multiple columns can be
added here when the target primary key demands.
4. Link the transformation to the target. Only link the required data columns from the
transformation to the target. No links will exist from the transformation to the target for
the CI and BI columns, nor for the DTL__CAPXACTION column unless required in
the target. This will result in the following mapping:
Note: In this example a datamap has been created for records with a REC_TYPE of “A” in the
example above, and for REC_TYPE of “B”. These data maps are ksdss1.ksdsm1 and
ksdss2.ksdsm2 respectively.
3. Now register each of these data maps for PowerExchange capture. This process is
described in the relevant PowerExchange Adapter Guide.
4. Assign the BI and CI PowerExchange fields to the extraction map as shown above in
“Configuring the PowerExchange Extraction Map” on page 177.
5. Now import these two extraction maps as data sources into the PowerCenter Designer
using Import from PowerExchange.
6. Import the relevant data targets. In this example the two input capture streams will be
written to two separate DB2 tables.
7. When the mapping is created it will look similar to the following:.
Note how the Flexible Transformation here contains the columns for both of the
PowerExchange change data input sources, and that they are linked to their own individual
output tables.
In step 7 on page 186 the Flexible Transaction includes Input and Output Groups. An input
and output group is added when the flexible transformation is created, but subsequent input/
output groups will be required for each source. These are added when you “Edit” the
transformation and select the “Ports” tab shown below and then use the Create Input Group
and Output Group buttons (the Create Input Group button is highlighted here):
Before validating the mapping the before imaged and change indicators will need to be
assigned to ports as shown in step 4 on page 185.
Chapter 8
PowerExchange ODBC -
Installation
This chapter covers the following topics:
♦ Overview, 190
♦ Creating ODBC Data Sources, 191
189
PWX_Interfaces_for_PowerCenter.book Page 190 Wednesday, October 3, 2007 9:53 AM
Overview
Before installing and configuring the PowerExchange ODBC connection, you must install
and configure PowerCenter and PowerExchange.
Note: PWXPC provides additional functionality that is not available with ODBC. See
“Functional Comparison between PWXPC and PowerExchange ODBC” on page 2.
Prerequisites
Before installing and configuring the PowerExchange ODBC connection, you must install
and configure PowerCenter and PowerExchange.
Installation Requirements
To use the PowerExchange ODBC connection with PowerCenter, the following products
must be installed:
♦ PowerCenter 8.5. For more information about installing PowerCenter see the
PowerCenter Installation Guide.
♦ PowerExchange 8.5. Install PowerExchange on the PowerCenter Client and Integration
Service machines. For more information about installing PowerExchange see the
PowerExchange Installation Guide.
The PowerCenter Client, Integration Service and Repository Server software needs to be
installed on the appropriate platforms. The PowerExchange software needs to be installed on
the same PowerCenter Client and Server machines.
If you have installed the 32-bit version of Integration Service, you must install the 32-bit
version of PWXPC and of PowerExchange. If you have installed the 64-bit version of
Integration Service, you must install the 64-bit version of PWXPC and of PowerExchange.
For more information about ODBC data source parameters (both mandatory and optional)
for a specific DBTYPE, see "Using ODBC with PowerExchange" in the PowerExchange
Reference Manual.
The <data source name> defined in the odbc.ini is specified in the Connect String value of
the ODBC Connection in PowerCenter. This connect string causes the PowerExchange
ODBC driver to be loaded and the specified location to be contacted to extract or load the
data. For more information on ODBC connectivity with PowerCenter, see the PowerCenter
Configuration Guide.
Chapter 9
PowerExchange ODBC -
Working with Mappings
This chapter covers the following topics:
♦ Overview, 194
♦ Working with Source and Target Definitions for PowerExchange Batch, 195
♦ Working with Source Definitions for PowerExchange Change or Real-Time, 198
193
PWX_Interfaces_for_PowerCenter.book Page 194 Wednesday, October 3, 2007 9:53 AM
Overview
A mapping is a set of source and target definitions linked by transformation objects that
define the rules for data transformation. Mappings represent the data flow between sources
and targets.
Source and target definitions represent metadata for sources and targets. When you create a
source definition, its structure differs depending on the type of source it represents:
♦ Non-relational sources require a multi-group source definition.
♦ Relational sources use a single-group source definition.
The source qualifier for a source definition also differs in structure depending on the type of
source definition.
After you create a source or target definition, you can include it in a mapping to extract data
from the source or load data to the target. You can extract source data in batch, change, or
real-time mode. You can use one source definition and one mapping for all modes.
For a list of sources and targets that PowerExchange ODBC Interface supports, see Table 1-4
on page 8. This table also lists whether the Integration Service can read the source data in
batch, change, or real-time mode.
Working with Source and Target Definitions for PowerExchange Batch 195
PWX_Interfaces_for_PowerCenter.book Page 196 Wednesday, October 3, 2007 9:53 AM
1. Click Sources > Import from Database … in the Source Analyzer if importing a source
definition. Click Targets>Import from Database … in the Target Designer if importing a
target definition.
The following Import Tables dialog box appears.
or
2. Click Connect.
Working with Source and Target Definitions for PowerExchange Batch 197
PWX_Interfaces_for_PowerCenter.book Page 198 Wednesday, October 3, 2007 9:53 AM
where:
♦ D - the default entry starts with D (any user-modified maps start with U)
♦ N - Database-specific identifier (e.g., 1 for DB2/390, 2 for IMS, 3 for DB2/400, etc.)
♦ instance - instance name chosen for the source registration
♦ regname - the registration name chosen for the source registration.
For example, DB2/390 extraction map name might be: d1dsn7.testdb2_KJM723TB.
When you import Change Data Capture source definitions, the ODBC data source must
have a DB Type of CAPX or CAPXRT. These DB Types instruct the Listener to select
extraction maps rather than data maps.
If the Listener pointed to by the ODBC data source is running with PowerExchange
Security (either SECURITY=(1,x) or SECURITY=(2,x) in the DBMOVER
configuration), then a valid user name and password must be provided.
2. Click Connect.
Chapter 10
PowerExchange ODBC -
Configuring Connections
This chapter covers the following topics:
♦ Overview, 202
♦ Working with Connections for PowerExchange ODBC, 203
201
PWX_Interfaces_for_PowerCenter.book Page 202 Wednesday, October 3, 2007 9:53 AM
Overview
Before PowerCenter can access a source or target in a session, you must configure connections
in the Workflow Manager. When you create or modify a session that reads from or writes to a
database, you can select only configured source and target databases. Connections are saved in
the repository. For PowerExchange ODBC, you configure relational database connections.
Application Connection/
Extraction Mode Relational Database Connection Type
Connection
Batch mode (non-relational data map) Relational Connection ODBC with NRDB and NRDB2 Data
Source
Change mode Relational Database Connection ODBC with CAPX Data Source
Real-time mode Relational Database Connection ODBC with CAPXRT Data Source
Note: For more information about the full range of database types that can be specified in
ODBC Data Sources, see the PowerExchange Reference Manual.
Datacom No No No
DB2 (DB2400C) No No No
IDMS No No No
IMS No No No
Table 10-3 shows the Connection Object Definition dialog box for an ODBC relational
connection and describes the connection attributes to configure for an ODBC relational
database connection:
Connection Required/
Description
Attribute Optional
Code Page Required Code page for the Integration Service to use to extract the data from the data
source.
Connection Optional Executes an SQL command with each database connection. Default is disabled.
Environment SQL
Transaction Optional Executes an SQL command before the initiation of each transaction. Default is
Environment SQL disabled.
Connection Retry Optional Number of seconds the Integration Service attempts to reconnect to the
Period database if the connection fails. If the Integration Service cannot connect to the
database in the retry period, the session fails.
Chapter 11
PowerExchange ODBC -
Sessions and Workflows
This chapter covers the following topics:
♦ Overview, 208
♦ Extracting Data from PowerExchange in Batch Mode, 209
♦ Extracting Data from PowerExchange in Change and Real-Time Mode, 213
♦ Loading Data to PowerExchange Targets, 215
207
PWX_Interfaces_for_PowerCenter.book Page 208 Wednesday, October 3, 2007 9:53 AM
Overview
After you create mappings in the PowerCenter Designer, you can create a session and use the
session in a workflow to extract, transform, and load data. You create sessions and workflows
in the Workflow Manager.
You can create a session in a workflow to extract data in batch, change, or real-time mode. You
determine how you want the Integration Service to extract the data when you configure the
session. You can also create a session to load data to a target.
After you configure the workflow, you must schedule the workflow. For more information
about creating, configuring, and scheduling workflows, see the PowerCenterWorkflow
Administration Guide.
Pipeline Partitioning
Depending on your source or target database, you can increase the number of partitions in a
pipeline to improve session performance. Increasing the number of partitions allows the
Integration Service to create multiple connections to sources and targets and process
partitions of data concurrently. While processing data, the Integration Service may process
data out of sequence due to the varying rates at which the partitions process data.
When you create a session in a workflow, the Workflow Manager validates each pipeline in the
mapping for partitioning. You can specify multiple partitions in a pipeline if the Integration
Service can maintain data consistency when it processes the partitioned data.
For more information about partitioning and a list of all partitioning restrictions, see the
PowerCenterWorkflow Administration Guide.
1. In the Task Developer, double-click a session with a non-relational source to open the
session properties.
2. Click the Sources view on the Mapping tab.
3. In the Reader field of the Readers settings, Relational Reader is automatically selected.
4. In the Connections Value field, select the non-relational ODBC connection (NRDB or
NRDB2).
5. In the Properties settings, configure the Owner Name attribute. For more information
about other Properties attributes, see the PowerCenter Workflow Administration Guide.
At minimum, the schema name of the PowerExchange data map (or the Owner name
displayed when the source mapping is edited) must be specified in order to correct
construct the SQL statement during execution. PowerExchange SQL Escape Sequences
can also be specified in this attribute field to override specifications in the data map. For
a complete list of the SQL Escape Sequences available, see the PowerExchange Reference
Manual.
The following example indicates how the Owner Name attribute can be configured to
provide the schema name for the source (seq) as well as an override for the physical file
name in the data map (dtldsn=new.dataset.name):
seq{dtldsn=new.dataset.name}
Note: SQL escape sequences and the Owner Name can be specified in any order in the
attribute field.
6. Click OK.
1. In the Task Developer, double-click a session with a relational source to open the session
properties.
Click the Sources view on the Mapping tab.
At minimum, the owner name of the source table must be specified in order to correct
construct the SQL statement during execution. Alternatively, the Owner Name can be
specified in the source mapping and, if done, would not be required here.
5. Click OK.
1. In the Task Developer, double-click a session with a relational source to open the session
properties.
2. Click the Sources view on the Mapping tab.
3. In the Reader field of the Readers settings, Relational Reader is automatically selected.
4. In the Connections Value field, select a connection which points to an ODBC data
source with DBType CAPX (for Change) or CAPXRT (for Real-Time) and the
appropriate Location value.
5. In the Properties settings, configure the Owner Name attribute. For more information
about other Properties attributes, see the PowerCenter Workflow Administration Guide.
At minimum, the schema name of the source extraction map must be specified in order
to correct construct the SQL statement during execution. This name is the first qualifier
of extraction map name shown in the PowerExchange Navigator. It can also be
determined by checking the Owner Name in the source mapping. PowerExchange SQL
Escape Sequences can also be specified in this attribute field to override specifications in
the data map. For a complete list of the SQL Escape Sequences available, see the
PowerExchange Reference Manual.
The following example indicates the manner in which the Owner Name attribute can be
configured to provide the schema name for the source (d6vsam) as well as an override for
the application name specified in the DBQual2/Application Name field in the ODBC
data source (dtlapp=new_appname):
{dtlapp=new_appname}d6vsam
Note: SQL escape sequences and the Owner Name can be specified in any order in the
attribute field.
6. Click OK.
1. In the Task Developer, double-click a session with a relational source to open the session
properties.
2. Click the Targets view on the Mapping tab.
Chapter 12
217
PWX_Interfaces_for_PowerCenter.book Page 218 Wednesday, October 3, 2007 9:53 AM
Overview
Recovery and restart need to be considered when designing and configuring sessions and
workflows using either PowerExchange Change or Real-Time. The considerations differ
depending upon whether the session uses PowerExchange Client for PowerCenter (PWXPC)
or PowerExchange ODBC. This is primarily due to the differences in where the restart
information is maintained.
With PowerExchange ODBC, the restart information is controlled and maintained on the
PowerExchange Listener platform in the internal change information file (CDEP) using the
application name specified in the connection.
After a session completes successfully, PowerExchange updates the CDEP with the ending
restart tokens.
The application name odbc_db2demo13ac was created using DTLUAPPL with RSTTKN
GENERATE. An example of the control cards that were used is given below:
mod APPL odbc_db2demo13ac DSN7 RSTTKN GENERATE
add rsttkn db2demo1
add rsttkn db2demo2
add rsttkn db2demo3
end APPL odbc_db2demo13ac
print appl odbc_db2demo13ac
You can use either ADD APPL or MOD APPL for a new application name as with MOD
APPL DTLUAPPL will create the application name if it does not already exist.
After the new restart tokens are provided through DTLUAPPL, the session using this
application name can be started and it will use this restart tokens.
Appendix A
221
PWX_Interfaces_for_PowerCenter.book Page 222 Wednesday, October 3, 2007 9:53 AM
In this example, the PowerEchange data map name is IMS.IMS1T01_IMSSEG1.If you use
the IMS database name for the Capture Registration Name as well, the resulting extraction
map name will be d2reconid.IMS1T01_IMSSEG1.
The map name for both the data map and extraction map are the same. Since the schema
name is not used in the source definition table name, the names will be exactly the same when
imported in Designer. As a result, only one will be able to be imported from the same
Location. So, there are two choices:
1. Use a different name for the capture registration so that a unique extraction map name is
created.
2. Use a different Location name to import data maps and extraction maps
If you want to use the same names for both the data map name and the capture registration
name, then you will need to use a different location name in the Import from PowerExchange
to place these two source definitions in different folders.
To help organize sources in Designer, use location names in the PowerExchange dbmover.cfg
that indicate the type of data you are retrieving. Creating multiple NODE= statements
(location names) for the same listener is perfectly acceptable. In this example, if you create a
NODE called CDCMAPS in the dbmover.cfg and use this as the Location when importing
the IMS extraction map, it will be stored in a sub- folder called IMS_CDCMAPS.
This strategy of separating extraction maps from regular relational and non-relational source
metadata can be a useful way of organizing source metadata in Designer.
For multi-record non-relational data map sources, there is a more complex form of the syntax:
<group name1>=<filter condition1>;<group name2>=<filter condition2>;...
This more complex form of the syntax allows you to use different filters for different record
types or the same filter for only some of the record types in a multi-record source. You can
also use the simple form with multi-record source data maps which then causes that filter to
be applied to all records.
The following example shows how to use the PowerExchange Change indicator columns
(DTL__CI_column) to filter changed data specifically, the change indicator for the
ACCOUNT field which is called DTL__CI_ACCOUNT:
In the following example, the multi-record VSAM source contains four records which each
have unique field names. The group names for the four records are:
V07A_RECORD_LAYOUT, V07B_RECORD_LAYOUT, V07C_RECORD_LAYOUT,
and V07D_RECORD_LAYOUT. The filter contains the group-name filter syntax in order
to filter data records for the first two records. No filtering is done on the other two records.
The filter specified is in the Filter Overrides attribute is:
V07A_RECORD_LAYOUT=V07A_RECORD_KEY=1;V07B_RECORD_LAYOUT=V07B_RECORD_KEY=2
Because there are four records in the multi-record data, there will be four SELECT statements
created by PWXPC. The SELECT statements for the two records specified in the Filter
Overrides attribute will also have WHERE clauses for their specific filters.
In this example, the DTLREXE command specifies mode=(job,wait) which means that the
DTLREXE will wait for the job to complete. This, in turn, will cause the session to wait until
this pre-session command completes. In the “Error Handling” section of the Config Object,
you can specify how to handle errors for pre-session commands in the On Pre-session command
task error field.
Note: Ensure that the JOB submitted through DTLREXE includes the appropriate DTLNTS
steps if WAIT mode is requested. The PowerExchange RUNLIB, in member DTLREXE,
contains sample JCL to be used with DTLREXE that includes the required DTLNTS steps.
If you are using a stand-alone command task to submit a batch JOB using DTLREXE, then
there are no session configuration options to check for success or failure. If you want to test
the status of the command task in the following session, you will need to use one of the task-
specific workflow variables available in the Workflow Manager; that is, either PrevTaskStatus
or Status. These variables can be used in link conditions to test the status of tasks in a
workflow. For example:
Link Condition
The link condition is created by double-clicking on the link between the DTLREXE
command task and the s_bulk_db2demo123_db2demoabc session to which it is connected.
This will invoke the Expression Editor which allows you to add the test to ensure that the
DTLREXE command task succeeded, as shown below:
For additional information on link conditions and the expression editor, refer to the
“Working with Workflows” chapter in the PowerCenter Workflow Administration Guide.
The following example show a CREATEFILE command for a new generation of GDG data
set my.gdg:
Note: When using this procedure for GDG data sets, the GDG base name specified must exist
and GDGLOCATE=Y must be specified in your PowerExchange DBMOVER configuration
file on the MVS platform.
When you run the workflow, the new generation of the GDG is created in addition to the
normal processing of the workflow. The allocation parameters used to create the data set are
specified in the DBMOVER configuration used by the PowerExchange Listener.
The CREATEFILE command has a number of parameters. Any number of these can be
specified. The parameters and their values are separated with a space in the command. The
parameters are:
UID MVS / AS400 only Userid. This parameter is required if your Listener is running with
user security (SECURITY=1 or 2).
PWD MVS / AS400 only Password for the userid specified in UID.
EPWD MVS / AS400 only Encrypted password for the userid specified in UID. Only one of
PWD or EPWD is required.
MODELDCB MVS only Model DSCB to be used for the file creation. Generally, this is only
required for GDG data sets which are not SMS-managed.
These value override the equivalent parameters specified in the DBMOVER configuration
file. Any values not specified use either those specified in DBMOVER or the standard
PowerExchange defaults. Refer to the “Configuration File Parameters” chapter of
PowerExchange Reference Manual for further information on these parameters.
Appendix B
235
PWX_Interfaces_for_PowerCenter.book Page 236 Wednesday, October 3, 2007 9:53 AM
Overview
PowerCenter uses the following datatypes when reading source data, transforming the data,
and writing target data:
♦ Native datatypes. Specific to the source and target databases or PowerExchange. Native
datatypes appear in source and target definitions.
♦ Transformation datatypes. Generic datatypes that appear in transformations. The
PowerCenter Server uses the datatypes to move data across platforms.
For more information about transformation datatypes, see the PowerCenter Designer Guide.
Relational Datatypes
PowerExchange Client for PowerCenter supports the same datatypes for DB2/390, DB2/400,
and DB2/UDB that PowerCenter supports for DB2. It also supports the same Oracle
datatypes that PowerCenter supports and the same SQL Server datatypes that PowerCenter
supports.
For more information about PowerCenter datatypes, see the PowerCenter Designer Guide.
Appendix C
Troubleshooting
243
PWX_Interfaces_for_PowerCenter.book Page 244 Wednesday, October 3, 2007 9:53 AM
Troubleshooting
When I go into Designer, I get messages about failures to load DLLs.
This can happen when PowerExchange Client for PowerCenter plug-ins are installed but
cannot be loaded for various reasons like incorrect releases of PowerExchange installed or
PATH problems. For more information, see KnowledgeBase Article # 15346.
I want to import a DB2/400 source definition, but need to determine the name of the DB2/400
database on the AS/400 machine.
Use the AS400 DSPRDBDIRE command to see a list of databases on the AS/400 machine.
The session failed with an error stating that the PowerExchange message repository cannot
be loaded.
You can receive this error on UNIX when there is no PWX_HOME environment variable set
to the PowerExchange installation directory. Set the PWX_HOME environment variable to
the PowerExchange installation directory.
I set the Idle Time session condition to -1. However, the session completed with the following
message: Idle Time limit is reached.
This can occur if EOF=Y is specified in the PowerExchange configuration file (dbmover.cfg)
CAPI_CONNECTION statement. When you set EOF=Y, PowerExchange returns an EOF
(which stops the session) when it reaches the end of the change stream as determined at the
time the session starts reading from it. As a result, the PowerCenter session completes instead
of continuing to run.
This message can also occur if the connection with PowerExchange is stopped using the
PowerExchange STOPTASK command.
This is a generic message indicating the PWXPC encountered an error. Review the session log
for other messages indicating what the problem is. If there are no other error messages in the
session log, check the PowerExchange logs on both the Integration Service platform and the
Listener platform.
I want to read all of the changes I've captured and have them be inserts into a staging area.
How do I do this?
When using PowerExchange ODBC to read captured changes, INSERT is the default
operation. If you want to apply the changes to the target using the same operation as done on
the source (INSERT, UPDATE, or DELETE), you need to explicitly include an Update
Strategy in the mapping to make this happen by testing the DTL__CAPXACTION field. In
the Update Strategy Expression field, you would code:
DECODE(DTL__CAPXACTION,'I',DD_INSERT,'U',DD_UPDATE,'D',DD_DELETE,DD_REJECT)
When using PWXPC, the DTL__CAPXACTION field is automatically acted upon when
processing changed data. If you want to have all changes processed as INSERTs regardless of
the DTL__CAPXACTION field, you must code an Update Strategy specifying DD_INSERT
in the Update Strategy Expression field.
Troubleshooting 245
PWX_Interfaces_for_PowerCenter.book Page 246 Wednesday, October 3, 2007 9:53 AM
Appendix D
PWXPC - Messages
247
PWX_Interfaces_for_PowerCenter.book Page 248 Wednesday, October 3, 2007 9:53 AM
Overview
The PowerExchange Client for PowerCenter issues informational messages about setup and
operation as well as messages when errors are detected. All of the PWXPC messages are
prefixed with PWXPC_ followed by a 5 digit message number.
Messages from PowerExchange and PowerCenter also display in conjunction with these
messages.
For information about PowerExchange messages, see PowerExchange Message Reference.
For information about PowerCenter messages, see PowerCenter Troubleshooting Guide.
PWXPC_10002 reader cannot get connection reference for the Source Qualifier instance
qualifier_name.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_10003 reader failed to initialize the source for the Source Qualifier instance
qualifier_name.
Cause: The Integration Service could not initialize the session.
Action: See the additional error message for more information.
Action: Informational.
PWXPC_10021 The schema is overwritten to [schema] in the session properties for the
Source Qualifier [qualifier_name]
Cause: The source schema in the source definition Metadata Extensions has been
overridden in the session properties to the value shown.
Action: Informational.
PWXPC_10022 The data map is overwritten to [map_name] in the session properties for the
Source Qualifier [qualifier_name]
Cause: The data map name in the source definition Metadata Extensions has been
overridden in the session properties to the value shown.
Action: Informational.
PWXPC_10023 reader prepared SQL statement [SQL_statement] for the group [group] in
Source Qualifier [qualifier_name] for partition partition_number
Cause: The prepared SQL statement for the source qualifier is shown.
Action: Informational.
PWXPC_10025 Cannot get the field projection information for the source source_name.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_10026 reader cannot get the source list for the Source Qualifier qualifier_name.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_10027 The Source Qualifier [qualifier_name] has more than one source instance
associated with it
Cause: Multiple source definitions are connected to the same source qualifier.
Action: Associate only one source definition with a source qualifier.
PWXPC_10028 reader cannot get the value of the attribute attribute_name for the Source
Qualifier qualifier_name.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_10029 reader cannot get the fields for the source source_name.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_10032 An invalid extraction map name was provided for the source source_name of
Source Qualifier qualifier_name.
Cause: The extraction map in the session properties for the specified source is
missing.
Action: Specify an extraction map in the session properties.
or
Cause: The extraction map name is not a valid extraction map name or is not the
correct name for the source.
Action: Specify a valid extraction map name.
PWXPC_10034 dispatcher cannot fetch changed data from the PowerExchange Listener.
Error code: return_code.
Cause: The Integration Service could not extract data from the PowerExchange
Listener.
Action: See the additional error message for more information.
PWXPC_10037 dispatcher cannot create the transaction statement. Error code: return_code.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_10039 dispatcher cannot bind the columns for the transaction statement. Error code
return_code.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
Action: Import the repository object from a new XML file. Avoid editing the XML file
before importing.
PWXPC_10045 The Extraction Map Name map_name is not valid for source source_name.
Cause: The specified extraction map is not an extraction map for the source in the
mapping.
Action: Specify an extraction map for the source in the mapping.
PWXPC_10054 dispatcher cannot read restart tokens from the restart token file file_name.
Cause: The Integration Service could not extract changed data from the source
because it could not read the restart tokens from the specified restart token
file.
Action: See the additional error message for more information.
PWXPC_10055 dispatcher cannot write restart tokens to the restart token file file_name.
Cause: The Integration Service could not extract changed data from the source
because it could not read the restart tokens from the specified restart token
file.
PWXPC_10057 dispatcher cannot bind the restart token control columns for the transaction
statement. Error code error_code.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_10066 reader reader failed to enable recovery for the Source Qualifier
qualifier_name.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_10067 dispatcher cannot read restart tokens from the recovery cache file.
Cause: The Integration Service cannot run the session in recovery mode, because the
recovery cache file was modified or deleted.
Action: Run the session again. Because the Integration Service may have committed
some of the data to the target before the initial session failed, the target
database may receive duplicate records.
PWXPC_10068 dispatcher cannot convert the restart tokens read from the restart token file
file_name.
Cause: You edited the restart token file, and may have added some invalid characters.
Action: Verify that the information in the restart token file is valid.
PWXPC_10069 dispatcher cannot convert the restart tokens read from the recovery cache
file.
Cause: The Integration Service cannot run the session in recovery mode, because the
recovery cache file may have been modified.
Action: Run the session again. Because the Integration Service may have committed
some of the data to the target before the initial session failed, the target
database may receive duplicate records.
PWXPC_10073 dispatcher cannot bind restart tokens. Error Code <error code>.
Cause: Internal error.
Action: Contact Informatica Global Customer Support.
PWXPC_11004 Session override IMS Unload file name [unload_file] is being used
Cause: The IMS Unload File Name session property specifies file name unload_file.
Action: Informational.
PWXPC_11005 dispatcher cannot open the restart token file [file_name]. System Error
message [message] code [errno]
Cause: PWXPC encountered an error attempting to open the restart token file,
file_name. System error message, message, and the system error number, errno,
indicates the failure reason.
Action: Correct the error indicated in the system error message and error code and
then restart the task.
PWXPC_11006 dispatcher failure writing to restart token file [file_name]. System Error
message [message] code [errno]
Cause: PWXPC encountered an error writing to the restart token file, file_name.
System error message, message, and the system error number, errno, indicates
the failure reason.
Action: Correct the error indicated in the system error message and error code and
then restart the task.
PWXPC_11007 dispatcher failure reading from restart token file [file_name]. System Error
message [message] code [errno]
Cause: PWXPC encountered an error reading from the restart token file, file_name.
System error message, message, and the system error number, errno, indicates
the failure reason.
Action: Correct the error indicated in the system error message and error code and
then restart the task.
PWXPC_11008 dispatcher failure deleting restart token file [file_name]. System Error
message [message] code [errno]
Cause: PWXPC encountered an error deleting the restart token file, file_name.
System error message, message, and the system error number, errno, indicates
the failure reason.
Action: Correct the error indicated in the system error message and error code and
then restart the task.
PWXPC_12008 Invalid column name [column_name] detected for current table in user SQL
Cause: Invalid column name detected for table in user SQL.
Action: Correct the SQL.
Index
A group source 5, 6, 69
CDC datamap 133, 161
access method See also extraction map source definitions
CAPX 6 CDC sessions
CAPXRT 6, 7 adding source 169
Application Multi-Group Source Qualifiers recovery example 172
See source qualifiers removing source 169
application name restart 115, 149, 163
restart points 115, 165 restart token file 160
stopping 114, 115, 168
CDEP
B restart 165
batch mode change data capture
configuring sessions 127 See change mode
Before image See also real-time mode
Flexible transactions 182 Change Indicator
bulk load Flexible transactions 182
configuring 85 change mode
Condense 5
configuring connections 115
C configuring sessions 133
code pages
CAPX See also PowerCenter Installation and Configuration
access method 6 Guide
CAPXRT configuring 241
access method 6, 7 supported code pages 241
CBLO compression
See constraint-based loading configuring 106
CDC data Condense
259
PWX_Interfaces_for_PowerCenter.book Page 260 Wednesday, October 3, 2007 9:53 AM
260 Index
PWX_Interfaces_for_PowerCenter.book Page 261 Wednesday, October 3, 2007 9:53 AM
I P
Idle Time pacing size
configuring 88, 92, 98, 103 configuring 107
IDMS pipeline partitioning
batch mode application connections 90 See also PowerCenter Workflow Administration Guide
change mode application connections 91 batch mode 208
IMS description 126, 208
batch mode application connections 90 loading to targets 208
change mode application connections 91 $PMRootDir
connection types 78 Cache 88, 92, 98, 103
datatypes 237 Restart 87, 91, 97, 102, 116, 152
real-time mode application connections 91 PowerExchange
performance 107
PowerExchange Change Data Capture
L Flexible transformations 180
PowerExchange Client for PowerCenter
loading
performance 3
constraints 134
PowerExchange Configuration File
logger token 161
dbmover.cfg 14
PWXPC
M error messages 248
metadata extensions
editing 50 R
viewing 49
Real-time Flush Latency
viewing for non-relational source definitions 58
configuring 117
real-time mode
N configuring Real-time Flush Latency 117
configuring sessions 133
non-relational source definitions recovery
editing 50 creating the tables 159
editing metadata extensions 50 enhanced restart 171
viewing data map details 48 example 172
viewing metadata extensions 58 PM_REC_STATE table 150
non-relational sources PM_RECOVERY table 149
configuring batch mode sessions 127 PM_TGT_RUN_ID table 149
non-relational target definitions state file 151
editing 50 tables 149
editing metadata extensions 50 relational source definitions
editing 37
relational sources
O configuring batch mode sessions 130
Oracle relational target definitions
connection types 78 editing 37
creating source definitions 30 relational targets
real-time application connections 97, 102 configuring sessions 140, 142
restart
$PMRootDir/Cache 88, 92, 98, 103
$PMRootDir/Restart 87, 91, 97, 102, 116, 152
application name 115, 165
Index 261
PWX_Interfaces_for_PowerCenter.book Page 262 Wednesday, October 3, 2007 9:53 AM
V
S VSAM
sequence token 161 batch mode application connections 90
sequential data sets change mode application connections 91
group source 68 connection types 78
session conditions datatypes 237
Idle Time 88, 92, 98, 103 extracting data from multiple files 127
Real-time Flush Latency 117 group source 68
sessions real-time mode application connections 91
overview 126, 208
source definitions
DB2 21 W
editing metadata extensions 50 workflows
editing, extraction maps 59 overview 126, 208
editing, non-relational 50
editing, relational 37
IMS 39
viewing metadata extensions 49
viewing, extraction maps 57
262 Index