Contents

Chapter 1. Tutorial: Designing your first Using the performance monitor. . . . . . . 40

job . . . . . . . . . . . . . . . . . 1 Running server jobs and parallel jobs . . . . . 41
Setting up the exercise . . . . . . . . . . .4 The Job Run Options dialog box . . . . . . . 41
Starting the Designer client . . . . . . . . .4 Parameters page . . . . . . . . . . . . 41
Lesson checkpoint . . . . . . . . . . .7 Limits page . . . . . . . . . . . . . 42
Setting up your project . . . . . . . . . . .7 General page . . . . . . . . . . . . . 42
Lesson checkpoint . . . . . . . . . . .8 Creating jobs by using assistants . . . . . . . 42
Creating a new job . . . . . . . . . . . .8
Lesson checkpoint . . . . . . . . . . .9 Chapter 3. Setting up your data
Adding stages and links to your job . . . . . .9 connections . . . . . . . . . . . . 43
Adding stages . . . . . . . . . . . . .9 Creating a data connection object . . . . . . . 43
Adding links . . . . . . . . . . . . . 11 Creating a data connection object manually. . . 43
Renaming stages and links . . . . . . . . 11 Creating a data connection object from a
Lesson checkpoint . . . . . . . . . . . 12 metadata import. . . . . . . . . . . . 45
Configuring your job . . . . . . . . . . . 12 Creating a data connection object from a stage. . 46
Configuring the data source stage . . . . . . 12 Using a data connection object . . . . . . . . 47
Configuring the Transformer stage. . . . . . 14 Using a data connection object with a stage . . 48
Configuring the target file stage . . . . . . 16 Using a Data Connection object for a metadata
Lesson checkpoint . . . . . . . . . . . 17 import . . . . . . . . . . . . . . . 49
Compiling your job. . . . . . . . . . . . 17
Lesson checkpoint . . . . . . . . . . . 17 Chapter 4. Defining your data . . . . . 51
Running your job and viewing results . . . . . 17 Table definition window . . . . . . . . . . 51
Running the job . . . . . . . . . . . . 17 General page . . . . . . . . . . . . . 51
Viewing results . . . . . . . . . . . . 19 Columns page . . . . . . . . . . . . 52
Lesson checkpoint . . . . . . . . . . . 20 Format page . . . . . . . . . . . . . 53
NLS page . . . . . . . . . . . . . . 54
Chapter 2. Sketching your job designs 21 Relationships page . . . . . . . . . . . 54
Getting started with jobs . . . . . . . . . . 21 Parallel page . . . . . . . . . . . . . 55
Creating a job . . . . . . . . . . . . 21 Layout page . . . . . . . . . . . . . 55
Opening an existing job . . . . . . . . . 22 Locator page . . . . . . . . . . . . . 55
Saving a job . . . . . . . . . . . . . 22 Analytical information page . . . . . . . . 56
Naming a job . . . . . . . . . . . . . 23 Importing a table definition . . . . . . . . . 56
Stages . . . . . . . . . . . . . . . . 23 Using the Data Browser . . . . . . . . . 57
Parallel job stages . . . . . . . . . . . 23 Sharing metadata between projects . . . . . . 58
Server job stages. . . . . . . . . . . . 24 Shared metadata. . . . . . . . . . . . 58
Mainframe job stages . . . . . . . . . . 24 Importing metadata to the shared repository . . 59
Naming stages and shared containers. . . . . 24 Creating a table definition from shared metadata 60
Links . . . . . . . . . . . . . . . . 25 Creating a table from a table definition . . . . 61
Linking parallel stages. . . . . . . . . . 25 Creating a table from a table definition . . . . 62
Linking server stages . . . . . . . . . . 26 Synchronizing metadata . . . . . . . . . 64
Linking mainframe stages . . . . . . . . 28 Managing shared metadata . . . . . . . . 65
Link ordering. . . . . . . . . . . . . 28 Manually entering a table definition . . . . . . 66
Naming links . . . . . . . . . . . . . 29 Creating a table definition . . . . . . . . 66
Developing the job design . . . . . . . . . 29 Viewing or modifying a table definition . . . . . 82
Adding stages . . . . . . . . . . . . 29 Editing column definitions . . . . . . . . 82
Moving stages . . . . . . . . . . . . 30 Deleting column definitions . . . . . . . . 82
Renaming stages. . . . . . . . . . . . 30 Finding column definitions . . . . . . . . 82
Deleting stages . . . . . . . . . . . . 30 Propagating values . . . . . . . . . . . 83
Linking stages . . . . . . . . . . . . 31 Stored procedure definitions . . . . . . . . . 83
Moving links . . . . . . . . . . . . . 31 Importing a stored procedure definition . . . . 83
Editing stages . . . . . . . . . . . . 32 The table definition dialog box for stored
Cutting or copying and pasting stages . . . . 37 procedures . . . . . . . . . . . . . 84
Pre-configured stages . . . . . . . . . . 37 Manually entering a stored procedure definition 85
Annotations . . . . . . . . . . . . . 38 Viewing or modifying a stored procedure
Using the Data Browser . . . . . . . . . 39 definition . . . . . . . . . . . . . . 87

© Copyright IBM Corp. 1997, 2014 iii

Chapter 5. Making your jobs adaptable 89 Chapter 9. Comparing objects . . . . 155
Adding parameters to your jobs . . . . . . . 90 Comparing objects in the same project . . . . . 157
Creating a parameter set . . . . . . . . . . 91 Comparing objects in different projects . . . . . 157
Adding environment variables to your jobs. . . . 92 Compare command line tool . . . . . . . . 157
Adding parameter sets to your jobs . . . . . . 93
Inserting parameters and parameter sets as Chapter 10. Searching and impact
properties . . . . . . . . . . . . . . . 93 analysis. . . . . . . . . . . . . . 161
Specifying values for a parameter set in a sequence
Find facilities . . . . . . . . . . . . . 161
job . . . . . . . . . . . . . . . . . 94
Quick find . . . . . . . . . . . . . 161
Advanced find . . . . . . . . . . . . 162
Chapter 6. Making parts of your job Impact analysis. . . . . . . . . . . . 166
design reusable . . . . . . . . . . . 95
Local containers . . . . . . . . . . . . . 95 Chapter 11. Sharing and moving your
Creating a local container. . . . . . . . . 95 designs . . . . . . . . . . . . . . 179
Viewing or modifying a local container . . . . 96
Importing objects . . . . . . . . . . . . 179
Using input and output stages . . . . . . . 96
Importing previously exported objects . . . . 179
Deconstructing a local container . . . . . . 97
Importing external function definitions . . . . 184
Shared containers . . . . . . . . . . . . 97
Importing web service function definitions . . 185
Creating a shared container . . . . . . . . 98
Importing metadata by using InfoSphere
Naming shared containers . . . . . . . . 99
Metadata Asset Manager . . . . . . . . 185
Viewing or modifying a shared container
Importing IMS definitions . . . . . . . . 186
definition . . . . . . . . . . . . . . 99
Exporting objects . . . . . . . . . . . . 188
Editing shared container definition properties . . 99
Exporting IBM InfoSphere DataStage
Using a shared container in a job . . . . . . 100
components . . . . . . . . . . . . . 188
Pre-configured components. . . . . . . . 102
Exporting from the export menu . . . . . . 189
Converting containers . . . . . . . . . . 103
Specifying job dependencies . . . . . . . 190
Using export from the command line . . . . 191
Chapter 7. Defining special dsexport command . . . . . . . . . . 192
components . . . . . . . . . . . . 105
Special components for parallel jobs . . . . . . 105 Chapter 12. Documenting your
Parallel routines . . . . . . . . . . . 105 designs . . . . . . . . . . . . . . 195
Custom stages for parallel jobs . . . . . . 107
Generating a job report . . . . . . . . . . 196
Special components for server jobs . . . . . . 122
Requesting a job report from the command line 197
Server routines . . . . . . . . . . . . 122
Custom transforms . . . . . . . . . . 127
Data elements . . . . . . . . . . . . 129 Chapter 13. Getting jobs ready to run 199
Special components for mainframe jobs. . . . . 133 Compiling server jobs and parallel jobs. . . . . 199
Mainframe routines . . . . . . . . . . 133 Compilation checks - server jobs . . . . . . 199
Machine profiles . . . . . . . . . . . 138 Successful compilation . . . . . . . . . 200
IMS databases and IMS viewsets . . . . . . 139 Compile from the client command line . . . . 200
Viewing generated OSH code . . . . . . . 202
Generating code for mainframe jobs . . . . . . 202
Chapter 8. Configuring your designs 143
Job validation . . . . . . . . . . . . 202
Configuring parallel jobs . . . . . . . . . 143
Code generation . . . . . . . . . . . 203
Specifying general options . . . . . . . . 143
Job upload . . . . . . . . . . . . . 203
Enabling runtime column propagation . . . . 145
JCL templates . . . . . . . . . . . . 203
NLS page . . . . . . . . . . . . . 145
Code customization . . . . . . . . . . 204
Setting runtime options for your job . . . . . 145
Compiling multiple jobs . . . . . . . . . . 204
Specifying default time and date formats . . . 146
Selecting a local message handler. . . . . . 146
Configuring server jobs . . . . . . . . . . 146 Chapter 14. Building sequence jobs 207
Specifying general options . . . . . . . . 146 Creating sequence jobs . . . . . . . . . . 208
Setting National Language Support (NLS) Specifying triggers. . . . . . . . . . . . 208
properties . . . . . . . . . . . . . 148 Trigger types . . . . . . . . . . . . 209
Optimizing job performance . . . . . . . 149 Trigger expression syntax . . . . . . . . 210
Configuring mainframe jobs . . . . . . . . 150 Restarting sequence jobs. . . . . . . . . . 212
Specifying general options . . . . . . . . 150 Creating parameters in your sequence jobs . . . 213
Specifying a job parameter in a mainframe job 151 Creating environment variables in your sequence
Controlling code generation . . . . . . . 152 jobs . . . . . . . . . . . . . . . . 214
Supplying extension variable values . . . . . 153 Sequence job properties . . . . . . . . . . 215
Configuring operational metadata . . . . . 153 General page . . . . . . . . . . . . 215

iv Designer Client Guide

 Parameters page . . . . . . . . . . . 216 Creating and editing configuration files . . . . 242
Job Control page . . . . . . . . . . . 217 Message Handler Manager . . . . . . . . . 243
Dependencies page . . . . . . . . . . 217 Using the Message Handler Manager . . . . 244
Sequence job activities . . . . . . . . . . 218 Message handler file format . . . . . . . 245
General activity properties . . . . . . . . 218 JCL templates . . . . . . . . . . . . . 245
End Loop activity properties . . . . . . . 219
ExecCommand activity properties . . . . . 219 Chapter 17. Creating repository tree
Exception activity properties . . . . . . . 220 objects . . . . . . . . . . . . . . 247
Job Activity properties . . . . . . . . . 220
Repository tree . . . . . . . . . . . . . 247
Nested condition activity properties . . . . . 221
Creating new objects . . . . . . . . . . . 248
Notification activity properties . . . . . . 222
Create a new object on startup . . . . . . 248
Oozie Workflow Activity properties . . . . . 223
Create a new object from the repository tree . . 248
Routine activity properties . . . . . . . . 224
Create a new object from the main menu . . . 248
Sequencer activity properties . . . . . . . 224
Create a new object from the toolbar . . . . 249
Start Loop activity properties . . . . . . . 226
Terminator activity properties . . . . . . . 230
User variables activity properties . . . . . . 231 Appendix A. Product accessibility . . 251
Wait-For-File activity properties . . . . . . 232
Appendix B. Contacting IBM . . . . . 253
Chapter 15. Job control routine . . . 235
Appendix C. Accessing the product
Chapter 16. Tools for managing and documentation . . . . . . . . . . . 255
administering jobs . . . . . . . . . 237
Intelligent assistants . . . . . . . . . . . 237 Appendix D. Providing feedback on
Creating a template from a job . . . . . . 237 the product documentation . . . . . 257
Creating a job from a template . . . . . . 238
Using the Data Migration Assistant . . . . . 238 Notices and trademarks . . . . . . . 259
Managing data sets . . . . . . . . . . . 239
Structure of data sets . . . . . . . . . . 240
Starting the Data Set Manager . . . . . . . 241 Index . . . . . . . . . . . . . . . 265
Data set viewer. . . . . . . . . . . . 241

Contents v