Professional Documents
Culture Documents
PWX 910HF2 Netezza UserGuide en
PWX 910HF2 Netezza UserGuide en
0 HotFix2)
User Guide
Informatica PowerExchange for Netezza User Guide Version 9.1.0 HotFix2 September 2011 Copyright (c) 2005-2011 Informatica. All rights reserved. This software and documentation contain proprietary information of Informatica Corporation and are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright law. Reverse engineering of the software is prohibited. No part of this document may be reproduced or transmitted in any form, by any means (electronic, photocopying, recording or otherwise) without prior consent of Informatica Corporation. This Software may be protected by U.S. and/or international Patents and other Patents Pending. Use, duplication, or disclosure of the Software by the U.S. Government is subject to the restrictions set forth in the applicable software license agreement and as provided in DFARS 227.7202-1(a) and 227.7702-3(a) (1995), DFARS 252.227-7013 (1)(ii) (OCT 1988), FAR 12.212(a) (1995), FAR 52.227-19, or FAR 52.227-14 (ALT III), as applicable. The information in this product or documentation is subject to change without notice. If you find any problems in this product or documentation, please report them to us in writing. Informatica, Informatica Platform, Informatica Data Services, PowerCenter, PowerCenterRT, PowerCenter Connect, PowerCenter Data Analyzer, PowerExchange, PowerMart, Metadata Manager, Informatica Data Quality, Informatica Data Explorer, Informatica B2B Data Transformation, Informatica B2B Data Exchange Informatica On Demand, Informatica Identity Resolution, Informatica Application Information Lifecycle Management, Informatica Complex Event Processing, Ultra Messaging and Informatica Master Data Management are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners. Portions of this software and/or documentation are subject to copyright held by third parties, including without limitation: Copyright DataDirect Technologies. All rights reserved. Copyright Sun Microsystems. All rights reserved. Copyright RSA Security Inc. All Rights Reserved. Copyright Ordinal Technology Corp. All rights reserved.Copyright Aandacht c.v. All rights reserved. Copyright Genivia, Inc. All rights reserved. Copyright Isomorphic Software. All rights reserved. Copyright Meta Integration Technology, Inc. All rights reserved. Copyright Intalio. All rights reserved. Copyright Oracle. All rights reserved. Copyright Adobe Systems Incorporated. All rights reserved. Copyright DataArt, Inc. All rights reserved. Copyright ComponentSource. All rights reserved. Copyright Microsoft Corporation. All rights reserved. Copyright Rogue Wave Software, Inc. All rights reserved. Copyright Teradata Corporation. All rights reserved. Copyright Yahoo! Inc. All rights reserved. Copyright Glyph & Cog, LLC. All rights reserved. Copyright Thinkmap, Inc. All rights reserved. Copyright Clearpace Software Limited. All rights reserved. Copyright Information Builders, Inc. All rights reserved. Copyright OSS Nokalva, Inc. All rights reserved. Copyright Edifecs, Inc. All rights reserved. Copyright Cleo Communications, Inc. All rights reserved. Copyright International Organization for Standardization 1986. All rights reserved. Copyright ej-technologies GmbH . All rights reserved. Copyright Jaspersoft Corporation. All rights reserved. This product includes software developed by the Apache Software Foundation (http://www.apache.org/), and other software which is licensed under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. This product includes software which was developed by Mozilla (http://www.mozilla.org/), software copyright The JBoss Group, LLC, all rights reserved; software copyright 1999-2006 by Bruno Lowagie and Paulo Soares and other software which is licensed under the GNU Lesser General Public License Agreement, which may be found at http:// www.gnu.org/licenses/lgpl.html. The materials are provided free of charge by Informatica, "as-is", without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability and fitness for a particular purpose. The product includes ACE(TM) and TAO(TM) software copyrighted by Douglas C. Schmidt and his research group at Washington University, University of California, Irvine, and Vanderbilt University, Copyright 1993-2006, all rights reserved. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit (copyright The OpenSSL Project. All Rights Reserved) and redistribution of this software is subject to terms available at http://www.openssl.org and http://www.openssl.org/source/license.html. This product includes Curl software which is Copyright 1996-2007, Daniel Stenberg, <daniel@haxx.se>. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://curl.haxx.se/docs/copyright.html. Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. The product includes software copyright 2001-2005 () MetaStuff, Ltd. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://www.dom4j.org/ license.html. The product includes software copyright 2004-2007, The Dojo Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://dojotoolkit.org/license. This product includes ICU software which is copyright International Business Machines Corporation and others. All rights reserved. Permissions and limitations regarding this software are subject to terms available at http://source.icu-project.org/repos/icu/icu/trunk/license.html. This product includes software copyright 1996-2006 Per Bothner. All rights reserved. Your right to use such materials is set forth in the license which may be found at http:// www.gnu.org/software/ kawa/Software-License.html. This product includes OSSP UUID software which is Copyright 2002 Ralf S. Engelschall, Copyright 2002 The OSSP Project Copyright 2002 Cable & Wireless Deutschland. Permissions and limitations regarding this software are subject to terms available at http://www.opensource.org/licenses/mit-license.php. This product includes software developed by Boost (http://www.boost.org/) or under the Boost software license. Permissions and limitations regarding this software are subject to terms available at http:/ /www.boost.org/LICENSE_1_0.txt. This product includes software copyright 1997-2007 University of Cambridge. Permissions and limitations regarding this software are subject to terms available at http:// www.pcre.org/license.txt. This product includes software copyright 2007 The Eclipse Foundation. All Rights Reserved. Permissions and limitations regarding this software are subject to terms available at http://www.eclipse.org/org/documents/epl-v10.php. This product includes software licensed under the terms at http://www.tcl.tk/software/tcltk/license.html, http://www.bosrup.com/web/overlib/?License, http://www.stlport.org/doc/ license.html, http://www.asm.ow2.org/license.html, http://www.cryptix.org/LICENSE.TXT, http://hsqldb.org/web/hsqlLicense.html, http://httpunit.sourceforge.net/doc/ license.html, http://jung.sourceforge.net/license.txt , http://www.gzip.org/zlib/zlib_license.html, http://www.openldap.org/software/release/license.html, http://www.libssh2.org, http://slf4j.org/license.html, http://www.sente.ch/software/OpenSourceLicense.html, http://fusesource.com/downloads/license-agreements/fuse-message-broker-v-5-3-licenseagreement; http://antlr.org/license.html; http://aopalliance.sourceforge.net/; http://www.bouncycastle.org/licence.html; http://www.jgraph.com/jgraphdownload.html ; http:// www.jcraft.com/jsch/LICENSE.txt. http://jotm.objectweb.org/bsd_license.html; http://www.w3.org/Consortium/Legal/2002/copyright-software-20021231; http://www.slf4j.org/ license.html; http://developer.apple.com/library/mac/#samplecode/HelpHook/Listings/HelpHook_java.html; http://www.jcraft.com/jsch/LICENSE.txt; http:// nanoxml.sourceforge.net/orig/copyright.html; http://www.json.org/license.html; http://forge.ow2.org/projects/javaservice/; http://www.postgresql.org/about/license.html; http:// www.sqlite.org/copyright.html; http://www.tcl.tk/software/tcltk/license.html; http://www.jaxen.org/faq.html; http://www.jdom.org/docs/faq.html; and http://www.slf4j.org/ license.html.
This product includes software licensed under the Academic Free License (http://www.opensource.org/licenses/afl-3.0.php), the Common Development and Distribution License (http://www.opensource.org/licenses/cddl1.php ) the Common Public License (http://www.opensource.org/licenses/cpl1.0.php ), the Sun Binary Code License Agreement Supplemental License Terms, the BSD License (http://www.opensource.org/licenses/bsd-license.php), the MIT License (http://www.opensource.org/licenses/mitlicense.php) and the Artistic License (http://www.opensource.org/licenses/artistic-license-1.0). This product includes software copyright 2003-2006 Joe WaInes, 2006-2007 XStream Committers. All rights reserved. Permissions and limitations regarding this software are subject to terms available at http://xstream.codehaus.org/license.html. This product includes software developed by the Indiana University Extreme! Lab. For further information please visit http://www.extreme.indiana.edu/. This product contains runtime modules of IBM DB2 Driver for JDBC and SQLJ (c) Copyright IBM Corporation 2006 All rights reserved. This Software is protected by U.S. Patent Numbers 5,794,246; 6,014,670; 6,016,501; 6,029,178; 6,032,158; 6,035,307; 6,044,374; 6,092,086; 6,208,990; 6,339,775; 6,640,226; 6,789,096; 6,820,077; 6,823,373; 6,850,947; 6,895,471; 7,117,215; 7,162,643; 7,254,590; 7,281,001; 7,421,458; 7,496,588; 7,523,121; 7,584,422, 7,720,842; 7,721,270; and 7,774,791 , international Patents and other Patents Pending. DISCLAIMER: Informatica Corporation provides this documentation "as is" without warranty of any kind, either express or implied, including, but not limited to, the implied warranties of noninfringement, merchantability, or use for a particular purpose. Informatica Corporation does not warrant that this software or documentation is error free. The information provided in this software or documentation may include technical inaccuracies or typographical errors. The information in this software and documentation is subject to change at any time without notice. NOTICES This Informatica product (the "Software") includes certain drivers (the "DataDirect Drivers") from DataDirect Technologies, an operating company of Progress Software Corporation ("DataDirect") which are subject to the following terms and conditions: 1. THE DATADIRECT DRIVERS ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. 2. IN NO EVENT WILL DATADIRECT OR ITS THIRD PARTY SUPPLIERS BE LIABLE TO THE END-USER CUSTOMER FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL OR OTHER DAMAGES ARISING OUT OF THE USE OF THE ODBC DRIVERS, WHETHER OR NOT INFORMED OF THE POSSIBILITIES OF DAMAGES IN ADVANCE. THESE LIMITATIONS APPLY TO ALL CAUSES OF ACTION, INCLUDING, WITHOUT LIMITATION, BREACH OF CONTRACT, BREACH OF WARRANTY, NEGLIGENCE, STRICT LIABILITY, MISREPRESENTATION AND OTHER TORTS. Part Number: PWX-NZU-91000-HF2-0001
Table of Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Informatica Resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Informatica Customer Portal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Informatica Documentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Informatica Web Site. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Informatica How-To Library. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii Informatica Knowledge Base. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Informatica Multimedia Knowledge Base. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Informatica Global Customer Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Table of Contents
Pipeline Partitioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Target Connection Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Multiple Targets Configuration for the Same Target Table. . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Netezza Target Data Update. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Update As Insert. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Update Else Insert. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Netezza Session Configuration for Optimal Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Netezza Distribution Key. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Troubleshooting Netezza Sessions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
ii
Table of Contents
Preface
The Informatica PowerExchange for Netezza User Guide provides information about extracting data from a Netezza source and loading data into a Netezza target. It is written for database administrators and developers who are responsible for extracting data from Netezza and loading data to Netezza. This book assumes you have knowledge of Netezza and PowerCenter.
Informatica Resources
Informatica Customer Portal
As an Informatica customer, you can access the Informatica Customer Portal site at http://mysupport.informatica.com. The site contains product information, user group information, newsletters, access to the Informatica customer support case management system (ATLAS), the Informatica How-To Library, the Informatica Knowledge Base, the Informatica Multimedia Knowledge Base, Informatica Product Documentation, and access to the Informatica user community.
Informatica Documentation
The Informatica Documentation team takes every effort to create accurate, usable documentation. If you have questions, comments, or ideas about this documentation, contact the Informatica Documentation team through email at infa_documentation@informatica.com. We will use your feedback to improve our documentation. Let us know if we can contact you regarding your comments. The Documentation team updates documentation as needed. To get the latest documentation for your product, navigate to Product Documentation from http://mysupport.informatica.com.
iii
Standard Rate Belgium: +31 30 6022 797 France: +33 1 4138 9226 Germany: +49 1805 702 702 Netherlands: +31 306 022 797 United Kingdom: +44 1628 511445
iv
Preface
CHAPTER 1
Code Pages
When the PowerCenter Integration Service runs in Unicode mode, it encodes Netezza data of the Nchar(m) and NVarchar(m) datatypes in UTF-8. It encodes Netezza data of the Varchar and Char datatypes in Latin-9. If the data contains extended ASCII characters or UTF-8 characters, run the PowerCenter Integration Service in Unicode mode.
CHAPTER 2
Prerequisites
Before you configure PowerExchange for Netezza, complete the following tasks:
Install the client and server components of the Netezza Performance Server. Verify that the Netezza database user has the following privileges on the database: - CREATE TABLE - CREATE EXTERNAL TABLE - DELETE - DROP - INSERT - LIST - SELECT - TRUNCATE - UPDATE
1. 2.
Upgrade PowerCenter. Configure the Repository Service to run in exclusive mode. To change the Repository Service operating mode, you can use the Administrator tool or the infacmd UpdateRepositoryService command.
3.
Use the pmrep UpgradeNetezzaToRelational command to upgrade PowerExchange for Netezza. Enter the following command:
pmrep upgradeNetezzaToRelational
4.
Note: If you do not have the correct privileges to register the plug-in, contact the user who manages the PowerCenter Repository Service.
Integration Service process runs. Use the Microsoft ODBC Data Source Administrator to configure ODBC connectivity.
PowerCenter Client. Install the Netezza ODBC driver on each PowerCenter Client machine that accesses the
Netezza database. Use the Microsoft ODBC Data Source Administrator to configure ODBC connectivity. Use the Workflow Manager to create a database connection object for the Netezza database.
6.
Verify that you can connect to the Netezza database. You can use the Microsoft ODBC Data Source Administrator to test the connection to the database. To test the connection, select the Netezza data source and click Configure. On the Testing tab, click Test Connection and enter the connection information for the Netezza schema.
Using a C shell:
$ setenv ODBCHOME =<Informatica server home>/ODBC6.1
PATH. Set the variable to the ODBCHOME/bin directory. For example: Using a Bourne shell:
PATH="${PATH}:$ODBCHOME/bin"
Using a C shell:
$ setenv PATH ${PATH}:$ODBCHOME/bin
NZ_ODBC_INI_PATH. Set the variable to point to the directory that contains the odbc.ini file. For example, if the odbc.ini file is in the $ODBCHOME directory: Using a Bourne shell:
NZ_ODBC_INI_PATH=$ODBCHOME; export NZ_ODBC_INI_PATH
Using a C shell:
$ setenv NZ_ODBC_INI_PATH $ODBCHOME
3.
Set the shared library environment variable. The shared library path must contain the ODBC libraries. It must also include the Informatica services installation directory (server_dir). Set the shared library environment variable based on the operating system. For 32-bit UNIX platforms, set the Netezza library folder to <NetezzaInstallationDir>/lib. For 64-bit UNIX platforms, set the Netezza library folder
to <NetezzaInstallationDir>/lib64. The following table describes the shared library variables for each operating system:
Operating System Solaris Linux AIX HP-UX Variable LD_LIBRARY_PATH LD_LIBRARY_PATH LIBPATH SHLIB_PATH
For HP-UX
Using a Bourne shell: $ SHLIB_PATH=${SHLIB_PATH}:$HOME/server_dir:$ODBCHOME/lib:<NetezzaInstallationDir>/lib64; export SHLIB_PATH Using a C shell: $ setenv SHLIB_PATH ${SHLIB_PATH}:$HOME/server_dir:$ODBCHOME/lib:<NetezzaInstallationDir>/lib64
For AIX
Using a Bourne shell: $ LIBPATH=${LIBPATH}:$HOME/server_dir:$ODBCHOME/lib:<NetezzaInstallationDir>/lib64; export LIBPATH Using a C shell: $ setenv LIBPATH ${LIBPATH}:$HOME/server_dir:$ODBCHOME/lib:<NetezzaInstallationDir>/lib64
4.
Edit the existing odbc.ini file or copy the odbc.ini file to the home directory and edit it. This file exists in $ODBCHOME directory.
$ cp $ODBCHOME/odbc.ini $HOME/.odbc.ini
Add an entry for the Netezza data source under the section [ODBC Data Sources] and configure the data source. For example:
[NZSQL] Driver = /export/home/appsqa/thirdparty/netezza/lib64/libnzodbc.so Description = NetezzaSQL ODBC Servername = netezza1.informatica.com Port = 5480 Database = infa Username = admin Password = password Debuglogging = true StripCRLF = false PreFetch = 256 Protocol = 7.0 ReadOnly = false ShowSystemTables = false
For more information about Netezza connectivity, see the Netezza ODBC driver documentation. 5. Verify that the last entry in the odbc.ini file is InstallDir and set it to the ODBC installation directory. For example:
InstallDir=/usr/odbc
6. 7.
Edit the .cshrc or .profile file to include the complete set of shell commands. Save the file and either log out and log in again, or run the source command. Using a Bourne shell:
$ source .profile
Using a C shell:
$ source .cshrc
CHAPTER 3
Source Filter
Description Number of columns used when sorting rows queried from the source. The PowerCenter Integration Service adds an ORDER BY clause to the default query when it reads source rows. The ORDER BY clause includes the number of ports specified, starting from the top of the transformation. When you specify the number of sorted ports, the database sort order must match the session sort order. Default is 0. Overrides the default query. Enclose column names in double quotes. The SQL query is case sensitive.
SQL Query
The source definition appears in the Source Analyzer. In the Navigator, the source definition appears in the Sources node of the active repository folder under the source database name.
If you need to create or modify a Netezza data source, click the Browse button to open the ODBC Administrator. Create the Netezza data source and click OK. Select the new Netezza data source. 3. Enter the user name and password to connect to the database, and click Connect. If you are not the owner of the table you want to use as a target, specify the owner name. 4. 5. Drill down through the list of database objects to view the available tables as targets. Select the relational table or tables to import the definitions into the repository. You can hold down the Shift key to select a block of tables, or hold down the Ctrl key to make nonconsecutive selections. You can also use the Select All and Select None buttons to select or clear all available targets. 6. Click OK. The selected target definitions appear in the Navigator under the Targets node.
CHAPTER 4
Normal Mode
In normal mode, the PowerCenter Integration Service extracts and loads data row by row. Run the session in normal mode if you want to use the following features:
Recovery Real-time sessions Pushdown optimization Commit interval Implicit join based on primary key and foreign key
Bulk Mode
You can transfer data in Netezza using bulk mode. Use bulk mode to increase session performance. In bulk mode, the PowerCenter Integration Service reads and writes Netezza data through an external table. An external table's definition is stored within the Netezza database but the data is saved externally in a location that is accessible to the Netezza host or the client system. Create external tables to structure your loading operation and manipulate data by using Netezza SQL. When the PowerCenter Integration Service extracts from Netezza, it creates an external table in the pipe directory path specified for extraction. When the PowerCenter Integration Service loads to Netezza, it creates an external table in the pipe directory path specified for loading. You can load data from the external table to the target. If
10
duplicate row handling is configured, data is loaded from the external table to a temporary table and then finally to the target. To transfer data in Netezza in bulk mode, complete the following steps: 1. 2. 3. 4. Verify that the Netezza database user has the LIST and CREATE EXTERNAL TABLE privileges on the database. Register the PowerExchange for Netezza plug-in with the repository. Configure the session to use a Netezza bulk reader and Netezza bulk writer. Configure the session properties as described in the following sections.
Required if the machine hosting the PowerCenter Integration Service is on HP-UX and the following directory is on an NFS-mounted directory:
<PowerCenter Installation Directory>/server/bin
Enter a path that does not use an NFS mount. Delimiter Delimiter separates successive input fields. You can enter any value supported by the Netezza Performance Server. The value can be a part of the data for the Netezza source. Default is |. NullValue parameter of an external table. The PowerCenter Integration Service uses the NullValue internally. Maximum value is one character. Default is blank. Escape character of an external table. If the data contains NULL, CR, and LF characters in the Char or Varchar field, you need to add an escape character in the source data before extracting. Enter an escape character before the data. The supported escape character is backslash (\).
NullValue
EscapeCharacter
Note: You can view load statistics in the session log. The load summary in the Workflow Monitor does not display load statistics.
11
Target Properties
You can configure the session properties for Netezza targets in the Transformations view on the Mapping tab. Define the properties for each target instance in the session. To load data in normal mode, configure the session to use a relational writer. To load data in bulk mode, configure the session to use a Netezza bulk writer. The session properties for normal mode are the same as that of any other relational target. The following table describes the session properties that you must configure to load Netezza target data in bulk mode:
Target Property Socket Buffer Size Description Set the socket buffer size to 25 to 50 % of the DTM buffer size to increase session performance. You might need to test different settings for optimal performance. Enter a value between 4096 and 2147483648 bytes. Default is 8388608 bytes. Path for the PowerCenter Integration Service to create the pipe for the external table. If you do not specify the path, the PowerCenter Integration Service uses the folllowing directory to create the pipe for the external table:
<PowerCenter Installation Directory>/server/bin
Required if the machine hosting the PowerCenter Integration Service is on HP-UX and the following directory is on an NFS-mounted directory:
<PowerCenter Installation Directory>/server/bin
Enter a path that does not use an NFS mount. Error Log Directory Name Error log directory can reside on the machine where the PowerCenter Integration Service runs. For example, you can use the following directory:
$PMBadFileDir
By default, the PowerCenter Integration Service creates the error log in the following directory on the machine hosting the Netezza Performance Server:
/tmp
The PowerCenter Integration Service creates a bad file in the error log directory if the data is not valid. Truncate Target Table Option The PowerCenter Integration Service truncates the target before loading. Run the truncate table command. Default is disabled. If you specify an SQL statement in the Pre-SQL property, the PowerCenter Integration Service runs the SQL statement before the table is truncated. Note: In normal mode, the PowerCenter Integration Service truncates the target after loading. Run the delete command. If you specify an SQL statement in the Pre-SQL property, the PowerCenter Integration Service runs the SQL statement after the table is deleted. You can override the default target table name. Set the delimiter to any value supported by the Netezza Performance Server. The delimiter separates successive input fields. The value must not be a part of the input data. Default is |.
12
Description CTRLCHARS parameter of the external table to transfer data containing control characters. You can enter control characters for Char and Varchar fields. If you enter a control character, you must add an escape character for the NULL, CR, and LF fields. Default is TRUE. CRINSTRING parameter to transfer data containing carriage returns (CR). You can enter a non escape CR in Char or Varchar fields. To load the control characters present in the Char and Varchar fields, set the CTRLCHARS and CRINSTRING parameters to TRUE in the session properties for the Netezza source. Default is TRUE. NullValue parameter of the external table. The PowerCenter Integration Service uses the NullValue internally. Maximum value is one character. Default is blank. Escape character of the external table. If the data contains NULL, CR, and LF characters in the Char or Varchar field, you need to add an escape character for these fields before loading. Enter a backslash (\) as the escape character. QUOTEDVALUE parameter of the external table. Select SINGLE or DOUBLE to enclose the field in single or double quotes. Select NO to omit quotes. Default is NO. The quoted value is not a part of the data. Ignores constraints on primary key fields. When you select this option, the PowerCenter Integration Service can write duplicate rows with the same primary key to the target. Default is disabled. The PowerCenter Integration Service ignores this value when the target operation is update as update or update else insert. Determines how the PowerCenter Integration Service handles duplicate rows. Select one of the following values: - First Row. The PowerCenter Integration Service passes the first row to the target and rejects the rows that follow with the same primary key. - Last Row. The PowerCenter Integration Service passes the last duplicate row to the target and discards the rest of the rows. Default is First Row. Adds an escape character to all the special characters in the data. The special characters include \n, \r, \0, delimiter, escape character, and null value character. Ensure that the value of the escape character is entered in the EscapeCharacter attribute.
CRINSTRING
NullValue
EscapeCharacter
Quoted Value
Unprojected Columns
When the PowerCenter Integration Service generates SQL to load to a Netezza target, it ignores target columns that are not connected in the mapping. If a default value is defined in Netezza for an unconnected column, Netezza updates or populates the column with the default value.
Pipeline Partitioning
You can increase the number of partitions in a pipeline to improve session performance. When you increase the number of partitions, the PowerCenter Integration Service can create multiple connections to sources and targets and process partitions of sources and target data concurrently.
13
The Netezza Performance Server divides data into data slices. In a partitioned session that reads data from Netezza, each partition reads a different data slice to prevent data duplication except in the following cases:
You enter an SQL override query for a partition. You enter different values for the source filter across partitions. You enter different values for the user-defined join across partitions.
partition. You cannot perform multiple updates, multiple deletes, or update and delete simultaneously on a Netezza target.
To avoid unpredictable session results, configure the session properties for insert, delete, update, and
duplicate row handling to have the same value for each partition.
If you run a partitioned session that joins multiple sources, link the first column in the Source Qualifier
transformation to a source column that represents data for the Netezza table with the best distribution in Netezza. This means that the Netezza table is more uniformly distributed across Snippet Processing Units (SPU) than other tables.
If you run a partitioned session with key constraints, only one partition shows load statistics.
Use the following rules and guidelines when you configure multiple targets in a target connection group to write to the same Netezza target table:
Target Load Type Insert Target Options Rules and Guidelines
Select the Ignore Key Constraints target property for insert targets.
Update
Use a maximum of one update table for any target. Do not use with delete tables. Use a maximum of one delete table for any target. Do not use with update tables.
Delete
14
Update As Insert
When you configure the session to update as insert rows, the PowerCenter Integration Service uses the following process to update target rows:
If the source key value matches a target key value, the PowerCenter Integration Service does not insert the
source row.
If the source primary key value does not exist in the target, the PowerCenter Integration Service inserts the
source row. The following table describes how the PowerCenter Integration Service updates the target:
Source Data Target Data Updated Target Data Comment
1,a,1a1
The source primary key is found in the target. The row is not inserted. 1,b,1b1 Inserts 1,b,1b1. The source primary key is found in the target. The row is not inserted. 1,c,1c1 1,c,1c1 The source primary key is found in the target. The existing row 1,c,1c1 is retained. No insert is required. Inserts 1,d,1d1. The source primary key is found in the target. The existing row 1,a,1a3 is retained. No insert is required.
1,b,1b1 1,a,1a2
1,c,1c1
1,d,1d1 1,a,1a3
Note: In the pair of values, the first two values are the primary key, for example 1 (primary key), a (primary key), 1a1. The session is configured to consider key constraints.
row. It updates with the first or last source row matched, based on how you configure duplicate row handling.
15
If the source primary key value does not exist in the target, the PowerCenter Integration Service inserts the
source row. The following table describes how the PowerCenter Integration Service updates the target:
Source Data 1,2 Target Data 1,6 Updated Target Data 1,2 Comment
Updates 1,6 with 1,2. The source primary key is found in the target. The target row is updated based on duplicate row handling to use first row. Updates 1,8 with 1,2. Duplicate row handling is configured to update with first source row. Subsequent target rows with primary key 1 are updated with first source row. Inserts 2,4. The source primary key is not found in the target. The row is inserted. Drops 2,5. The source primary key is found in the target, and first duplicate row has been updated in the target.
1,3
1,8
1,2
2,4
2,4
2,5
3,7
3,7
Note: In the pair of values, the first value is the primary key, for example 1 (primary key), 2.
session.
Ignore key constraints. You can improve session performance by ignoring key constraints when writing to
Netezza targets. Since Netezza does not enforce key constraints, the PowerCenter Integration Service performs additional processing when a session that writes to Netezza requires key constraints.
16
For example, to obtain optimal performance for 3 million rows and 32 KB row size, set the following parameters:
Default buffer block size: 1,280,000 Line sequential buffer length: 202,400 Commit interval: 200,000 DTM buffer size: 28,000,000 Socket buffer size: 8388608 bytes Escape characters: None Ignore key constraints: Selected
A Netezza Reader session can stop responding if the source contains special characters like delimiter. Add escape characters in the session to eliminate the delimiters. This is a Netezza issue and the reference number is SWS-40577.
Netezza drivers 3.1.2 or 3.1.4 are used.
Multi-pipe and multi-partition sessions can stop responding randomly with Netezza drivers 3.1.2 or 3.1.4 on HPUX. Use the Netezza driver 4.04 P2 to avoid this issue.
The pipe directory path on HP-UX is on an NFS-mounted drive.
If the pipe directory path on HP-UX is on an NFS-mounted drive, Netezza writer sessions can stop responding or terminate unexpectedly. Specify a non NFS-mounted drive (for example, /tmp) in the pipe directory path of the Netezza Writer session property.
The environment variables are set incorrectly.
Check whether the environment variables PATH, LIBPATH, ODBCINI, and NZ_ODBC_INI_PATH are set correctly.
The permissions for the file paths in the session properties are set incorrectly.
Ensure that all the file paths in the session properties set for Netezza reader and writer sessions are correct and have proper permission. Check out for the ones which need directory path specification. If the issue persists, you can try killing the blocking Netezza sessions or disable Netezza ODBC tracing and ODBC tracing.
17
18
APPENDIX A
Datatype Reference
This appendix includes the following topic:
Netezza and Transformation Datatypes, 19
based on ANSI SQL-92 generic datatypes, which the PowerCenter Integration Service uses to move data across platforms. They appear in all transformations in a mapping. When the PowerCenter Integration Service reads source data, it converts the native datatypes to the comparable transformation datatypes before transforming the data. When the PowerCenter Integration Service writes to a target, it converts the transformation datatypes to the comparable native datatypes. The following table lists the Netezza datatypes that PowerCenter supports and the corresponding transformation datatypes:
Netezza Datatype BigInt Range Transformation Datatype Bigint Range
From -9,223,372,036,854,775,808 through 9,223,372,036,854,775,807 Precision of 19, scale of 0 Integer value Precision 1
Bool
True or false, on or off, 0 or 1, yes or no Precision 3, scale 0 Single character ANSI SQL date
String
Precision 5, scale 0 From 1 through 104,857,600 characters Jan. 1, 0001 A.D. to Dec. 31, 9999 A.D. (precision to the nanosecond) Precision 15 Precision 15
Float8 Float4
Double Double
19
Range
Range
Precision 10, scale 0 Single character Used for storing UTF-8 data BVarchar (length) Non-blank-padded string, variable storage length Used for storing UTF-8 data Numeric (precision, decimal), arbitrary precision number Precision must be between 1 and 38 Precision 6, scale 0
NVarchar(m)
String
Numeric
Decimal
Real
Real
Precision of 7, scale of 0 Double-precision floating-point numeric value Precision 5, scale 0 Jan. 1, 0001 A.D. to Dec. 31, 9999 A.D. (precision to the nanosecond) Jan. 1, 0001 A.D. to Dec. 31, 9999 A.D. (precision to the nanosecond) From 1 through 104,857,600 characters
SmallInt Time
Timestamp
Date/Time
Varchar
String
20
INDEX
D
databases connecting to Netezza (UNIX) 4 connecting to Netezza (Windows) 3 datatypes PowerExchange for Netezza 19 default values Netezza targets 13
P
partitioning Netezza sessions 13 pipe directory path setting 11 setting for HP-UX 11 plug-ins registering for Netezza 3 prerequisites Netezza installation 2
E
empty strings in Netezza 13
H
HP-UX pipe directory path, setting 11
S
socket buffer size target property 11, 12 Source Qualifier Netezza, overview 7
I
installation Netezza prerequisites 2
T
target connection groups using with Netezza 14 target property socket buffer size 11, 12 targets Netezza default values 13 unprojected columns in Netezza 13 using multiple for the same Netezza table 15
K
key constraints example 15
M
multiple targets for the same Netezza table 15
U
update as insert description for Netezza 15 update else insert description for Netezza 15 update strategy example 15 update as insert for Netezza 15 update else insert for Netezza 15 upgrading PowerExchange for Netezza 3
N
Netezza connecting from an integration service (Windows) 3 connecting from Informatica clients(Windows) 3 connecting to an Informatica client (UNIX) 4 connecting to an integration service (UNIX) 4 Netezza target connection groups using multiple targets for the same table 14
21