Document: Oracle Data Integrator 11g (11.1.

1) Incremental Meta Data load to Planning using ODI

Description:
(This document describes the way to automate the metadata load to the Hyperion Planning through Oracle Data integrator packages.)

History:
Version Description Change Author Publish Date

0.1 0.1

Initial Draft Review 1st

Gourav Atalkar Amit Sharma

23-July-2011 23-July-2011

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

Table of contents
1) Introduction 2) Requirements 3) Creating Planning Data Server physical Schema and Logical schema for using as Target 3 4 5

4) Creating MS SQL server Data Server Physical Schema and Logical Schema for using as source 13 5) Setting up the Change data Capture on Source 6) Creating Interface 7) Creating packages 18 20 24

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

Introduction
Consistent Set Journalizing provides the guarantee that when you have an ORDER_LINE change captured, the associated ORDER change has been also captured, and vice versa. Note that consistent set journalizing guarantees the consistency of the captured changes. The set of available changes for which consistency is guaranteed is called the Consistency Window. Changes in this window should be processed in the correct sequence (ORDER followed by ORDER_LINE) by designing and sequencing integration interfaces into packages. Although consistent set journalizing is more powerful, it is also more difficult to set up. It should be used when referential integrity constraints need to be ensured when capturing the data changes. For performance reasons, consistent set journalizing is also recommended when a large number of subscribers are required. Setting up Journalizing This is the basic process for setting up CDC on an Oracle Data Integrator data model. Each of these steps is described in more detail below. 1. 2. 3. 4. 5. Set the CDC parameters Add the datastores to the CDC For consistent set journalizing, arrange the datastores in order Add subscribers Start the journals

To set the data model CDC parameters: This includes selecting or changing the journalizing mode and journalizing knowledge module used for the model. If the model is already being journalized, it is recommended that you stop journalizing with the existing configuration before modifying the data model journalizing parameters. 1. Edit the data model you want to journalize, and then select the Journalizing tab. 2. Select the journalizing mode you want to set up: Consistent Set or Simple. 3. Select the Journalizing KM you want to use for this model. Only knowledge modules suitable for the data model's technology and journalizing mode, and that have been previously imported into at least one of your projects will appear in the list.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

4. Set the Options for this KM. Refer to the knowledge module's description for more information on the options. 5. Click OK to save the changes.

1) Requirement
We have a data store of some description and the data gets automatically incremented, instead of having to load all the data into the database at a set interval you can use ODI to capture the additional data.

We Will use Consistent set change data capture for incremental data load

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

2)Creating Planning Data Server physical Schema and Logical schema for using as Target:
Step: 1) Go to StartProgram FilesOracleOracle Data Integrator explore ODI and login to work repository.

Step: 2) After successfully login to work repository go to Topology Navigator select Physical Architecture Tab and select Technology and expand it. Go to the Hyperion Planning Right Click and select New Data Server.

Step: 3) In the definition tab enter the following details as shown in the picture below:

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

Step: 4) Click on Save it will ask you to create physical schema press OK.

Step:5) In Technology Tab go to Hyperion Planning expand it and select newly create data server Right Click and select New Physical schema.

Step:6) In the definition tab enter the Planning Application name to which you want to connect.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

Step: 8) Click on Save it will ask you to specify the context press OK.

Step: 9) Go to the Logical Architecture tab Expand Technologies and select Hyperion Planning Right-click and select New Logical Schema.

Step: 10) Give appropriate name to logical schema and set the Context to Hyperion planning physical schema.

Step: 11) Click on Save. Step: 12) Go to the ODI Designer and create new Project with appropriate name.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

Step: 13) Click on Save. You can check in the project tab

Step: 14) Expand project go to knowledge module right click and import following knowledge modules. RKM Hyperion Planning LKM File to SQL IKM SQL to Hyperion Planning

Step: 15) Press Ctrl and select all three knowledge

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

Step: 16) Save.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

10

Step: 17) Now go to the model tab and create new model.

Step: 18) In the definition tab enter the details

Step:19) Go to the Reverse Engineer tab select customize check box and enter details as follows:

Step: 20) Save Step:21) Newly created Model will be available for Reverse Engineering go to Model tab select Hyperion planning model and Right Click and select Reverse Engineers. Before Reverse Engineer the models make sure your agent services are running if not start it with following command: C:\oracle\Middleware\Oracle_ODI1\oracledi\agent\bin>agent NAME=localagent

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

11

Step: 22) Now you can reverse engineer it.

Press OK. Step: 23) Go to the Operator Tab check whether the Execution run successfully or not.

If all the checks are green that means your model reversed successfully.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

12

Step: 24) Go to the Designer and open model tab expand Hyperion planning model again. You can check all the dimensions are visible there from your planning application.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

13

3)Creating MS SQL server Data Server Physical Schema and Logical Schema for using as source
Step: 1) Now we need to create data server for MS SQL RDBMS these are the step to create it. Go to the Topology Navigator in Physical architecture expand Technology tab select Microsoft SQL Server Right Click and select new data Server. Specify following details: Name: Appropriate name Instance: bispfdm[server_name] User: sa Password: ******

Step 7) Go to the JDBC tab and set appropriate drivers and url for the SQL Server database. Test the connection by pressing the Test connection button.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

14

Step: 8) Save it.

Step: 9) In Technology Tab go to SQL Server expand it and select newly create data server Right Click and select New Physical schema. Select schema in which your database located.

Step: 10) Click on Save it will ask you to specify the context press OK.

Step: 11) Go to the Logical Architecture tab Expand Technologies and select Oracle Right-click and select New Logical Schema.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

15

Step: 12) Click on Save. Step: 13) Now go to Designer Navigator and then model tab and create new model Folder.

Step: 14) Save it. Step: 15) Now go to the newly created folder and create new model.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

16

Step: 16) In the definition tab enter the details.

Step:17) Save the model it will visible in your model folder now select it Right click and select reverse engineer.

Step: 19) Go to the Designer and open model tab expand Oracle model again. You can check all the tables metadata are visible there from your Oracle schema.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

17

Now the Source is ready to set for loading the data. Acc1 table has the parent/child/alias details and a primary key of ID.

The Acc2 table holds the member properties.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

18

4)

Setting up the Change data Capture on Source:

This time select SRC_Essbase _RDBMS data model open it and go to consistent set option under the Journalizing tab.

Save it. Next step is to add both the tables to use CDC, right click on the datastore and select.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

19

The session creates the tables, views and triggers required for journalizing. Tables are created which will hold the information about the changes. J$Acc1 and J$Acc2

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

20

5)Creating Interface
The next stage is to set up the interface that will load the changes into planning. Now the journalizing is active an interface can be created which will take the new records and load them into our essbase database. Step:1) Create interface

Step: 2) The two data stores are dragged onto the source, and the join between the two tables is automatically created specify the source and target tables and map them.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

21

Step: 3) Just like with simple journalizing you need to tick the Journalized data only box which creates three extra journal fields to the datastore (JRN_SUBSCRIBER, JRN_FLAG, JRN_DATE)

Step: 4) Our target is not capable of performing Transformation so we need to change the Staging area from target to another server we will select memory engine

Step: 5) Move to the Flow tab and specify the required knowledge modules.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

22

After inserting a record into the monitored datastores you can view the changes from ODI using Changed Data Capture > Consumption > Extend Window

Step: 6) Executing the interface will load any updates straight to planning, to reset the journal capturing after you have run the interface can be achieved by using Changed Data Capture > Consumption > Purge Journal

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

23

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

24

6)Creating packages
Step: 1) Create a package inside the project w have created previously.

Step: 2) Drag and drop waitForLogData Api tool from Change Data Capture and update the require field. The only difference being that with consistent set journalizing you cant use the Table Name parameter and have to use the CDC Set parameter.

In the CDC Set parameter it needs to be the name of your logical schema and model name, this can be retrieved by using an ODI API function, the code needs to be set as :<%=odiRef.getObjectName("L","model_code","logical_schema", "D")%> In my example the logical schema is called appodi application and the model code is HYPERION_PLANNING_MODEL so the API function required to be updated to:-

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

25

Now to perform the equivalent function of Extend Window, drag the model onto the diagram and set the type to Journalizing Model and check the Extend Window and Lock Subscriber box.

Step: 3) We need to drag and drop the interface to diagram tab and specify the sequence of execution.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

26

We need to reset the journalizing by using the purge command, this can be done like before by dragging the model on to the diagram but this time making sure the Purge Window and Unlock Subscriber box is checked.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

27

Finally we generate scenario and added the OdiStartScen utility to the diagram, this executes the scenario again which is a much more cleaner and sensible way than just looping back to the first step, if you loop straight back it can cause problems such as the logs growing out of control. I regenerated the scenario so it took into account the last step.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

28

Once the scenario is executed it will wait until 1 record have been updated and then run the journalizing process that captures the records, then the interfaces are run to load metadata into planning, finally the journals are purged and the process will be started again.

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

29

Learnhyperion.wordpress.com | learnodi.wordpress.com | http://odinetwork.blogspot.com/

30