Transformer Advanced

Advanced Training, Corporater Transformer
CORPORATER Transformer
Advanced Training
Last updated Dec. 2013 1

Contents
EPM Data model ...................................................................................................................................... 4
Transformer EPM ETL Tool ................................................................................................................... 5
1. Building a Transformer Project........................................................................................................ 6
1.1 Interface .................................................................................................................................. 6
1.1.1 Transformer model.......................................................................................................... 7
1.1.2 Transformer design ......................................................................................................... 7
1.1.3 Properties ........................................................................................................................ 8
1.1.4 Resources ........................................................................................................................ 8
1.1.5 Preview and progress ...................................................................................................... 8
1.2 Transformer Project Group ..................................................................................................... 9
1.2.1 Transformer Project ........................................................................................................ 9
1.2.2 Logs .................................................................................................................................. 9
2. Sources in Transformer ................................................................................................................. 10
2.1 SQL Source ............................................................................................................................. 10
2.2 HTTP source ........................................................................................................................... 12
2.3 MDX source ........................................................................................................................... 13
2.4 Excel and OpenOffice Calc sources ....................................................................................... 14
2.5 CSV source ............................................................................................................................. 15
2.6 Surveyor source ..................................................................................................................... 16
2.7 Surveyor web service source ................................................................................................. 17
2.8 LDAP source........................................................................................................................... 18
3. Processes in Transformer .............................................................................................................. 20
3.1 Aggregation ........................................................................................................................... 20
3.2 Copy ....................................................................................................................................... 22
3.3 Merger ................................................................................................................................... 23
3.4 Hierarchy for nodes and organizations ................................................................................. 24
3.5 Set constant date .................................................................................................................. 27
3.6 Set date ................................................................................................................................. 28
3.7 String to date conversion ...................................................................................................... 29
3.8 Calculation ............................................................................................................................. 30
3.8.1 NUM .............................................................................................................................. 30
3.8.2 STR ................................................................................................................................. 31
3.8.3 TRIM .............................................................................................................................. 32

3.8.4 REPLACE ......................................................................................................................... 32

3.8.5 LEN ................................................................................................................................. 34
3.8.6 SUBSTRING .................................................................................................................... 35
3.8.7 ABS................................................................................................................................. 36
3.8.8 ROUND........................................................................................................................... 37
3.8.9 FIND ............................................................................................................................... 39
3.8.10 NOP................................................................................................................................ 40
3.8.11 DATETONUMBER ........................................................................................................... 41
3.9 DePivot .................................................................................................................................. 42
3.10 Splitter ................................................................................................................................... 43
3.11 Node and Organization mapping .......................................................................................... 45
3.12 Multi calculation .................................................................................................................... 46
4. Targets in Transformer .................................................................................................................. 47
4.1 CSV target .............................................................................................................................. 47
4.2 EPM Data Records target ...................................................................................................... 48
4.3 Excel target ............................................................................................................................ 49
4.4 Trash can ............................................................................................................................... 49
4.5 Node target ........................................................................................................................... 50
4.6 Organization target ............................................................................................................... 51
4.7 Group target .......................................................................................................................... 52
4.8 User target............................................................................................................................. 53
4.9 Membership target ............................................................................................................... 54
5. Transformer Scheduler .................................................................................................................. 55
6. Validating imported data .............................................................................................................. 56
7. Exercises ........................................................................................................................................ 58
Suggested solution Exercise 1: Organization hierarchy and loading of organizations ...................... 59
Suggested solution Exercise 2: Node hierarchy and loading of nodes.............................................. 69
Suggested solution Exercise 3: Loading financial and budget values ............................................... 76

EPM Data model
To be able to load data into EPM it is important to know how the data is stored in EPM. The
EPM data model consists of four dimensions: Node, Node type, Period, and Organization. All
these four dimensions must be present to either load data into EPM or pull data from EPM.
Node: the reference to where the data is stored, the data holder. Nodes are the account
structure in EPM.
Node type: defines what type of data we want to use. The four default node types are
Actual, Budget, Target, and Forecast.
Period: for what period the data is stored.
Organization: what organization the data is for.

Transformer EPM ETL Tool
Transformer is a powerful ETL tool and is the module in EPM that is most commonly used for
loading data into the EPM model. ETL stands for extract, transform, and load. Transformer
allows you to import data from a wide range of data sources, modify the data to fit into
EPM, and load data into EPM.
Furthermore, Transformer allows you to load organization and node structures, in addition
to users from Active Directory.
This document includes a full description of the Transformer module including numerous
examples.

1. Building a Transformer Project
1.1 Interface
The shortcut to the Transformer module is located in the menu bar in Configuration Studio.
See figure 1.1.1.
Figure 1.1.1
The default Transformer module view consists of five windows: Transformer model,
Transformer design, Properties, Resources, and Preview and Progress.
Figure 1.1.2

1.1.1 Transformer model

The Transformer model window holds all your Transformer projects groups, and it is in this
window you create all your projects and set global properties for the different Transformer
projects and project groups. You add a new Transformer project by first creating a category
folder and adding a new Transformer project group to it. A Transformer project group can
hold multiple Transformer projects.
Figure 1.1.1.1
1.1.2 Transformer design

It is in this window you build your Transformer Project and manipulate your data stream.
You add new project objects by right clicking in the design window and choosing a source,
process, or target object. There is no practical limitation on how many objects you can add.
You connect your objects by dragging a line from one objects outlet (right side) to another
objects socket (left side), represented by a small circle on each side of an object. See figure
1.1.2.1
Figure 1.1.2.1

1.1.3 Properties
The Properties window contains the configuration for the chosen object. You can use this
window to change an objects attributes, or double click on the object in Transformer
Design.
1.1.4 Resources
The Resources view holds all the files and connections that are available for you to use when
configuring your Transformer project source object. Files and connections are made
available by uploading them to Resources.
1.1.5 Preview and progress

In Transformer you are able to preview your data stream on each object in a project. This
means that you can configure your object and preview the result of that object by simply
highlighting it in the Transformer Design and pressing the green Refresh icon in the top right
corner of the Preview and Progress window.
When you run a Transformer project the progress and result will be displayed here.
Figure 1.1.5.1

1.2 Transformer Project Group

When you create a new Transformer project group a Transformer project and a Log file are also
created by default. Additional projects and logs may be added manually.
1.2.1 Transformer Project

A Transformer project is where the actual transformation and loading of data is specified. A project
consists of a source, a target and one or more processes that are configured and connected in a
certain order. A Transformer project must at least contain a source and a target.
1.2.2 Logs
In Logs you will be able to view the logs that are created when a Transformer project group
is run. You can also add Log objects to each individual Transformer project; they will hold
only the log information for that specific project.

2. Sources in Transformer
A Transformer source is a file or other data resource that contains the data to be used in the
Transformer project. A file or database connection must be uploaded to Resources before it can be
used in a Transformer project.
2.1 SQL Source

An SQL source is used to retrieve data from SQL-based databases. To be able to connect to a
database, we first need to configure a connection to the database through an SQL Resource
in our Resources category. We will then use that connection to retrieve the data in the SQL-
source in Transformer. See figure 2.1.1.
Figure 2.1.1
Database type: Select the database type. The three most common SQL databases are
SQL server, Oracle, and MySQL, but Transformer supports all types of SQL Servers.
Server: Specify the IP address or hostname of the server where your SQL instance
resides.
Port: Specify which port your SQL instance listens to. This is usually 1433.
Instance: If you are running multiple SQL instances on the same server you need to
specify which instance to connect to. If you only run one instance you may leave this
blank.
Database: Specify the name of the database to connect to.

User name and Password: Specify the User name and Password to use to connect to the
database. The user must be a local SQL user who has access to the resources you are
going to extract data from.
You can test your connection by pressing the green refresh icon.
After testing the SQL resource add an SQL source to your project and choose the SQL
connection you want to use.
Figure 2.1.2
You can choose to use either an available table or an SQL expression to retrieve data from
your SQL source. You can also narrow down the time interval for the data by choosing a date
column in the source (if available) and using the date calculator to determine what interval
you want to use.

2.2 HTTP source

You can use an HTTP source to retrieve data stored on a webpage. You need to specify the
URL to the webpage and the character that separates the data. See Figure 2.2.1.
Figure 2.2.1

2.3 MDX source

MDX stands for Multidimensional Expressions which is a query language for Online Analytical
Processing Databases (OLAP databases).
First we need to create an MDX resource to connect to an OLAP database. We do this by

adding an MDX resource in our Resources catalogue.
Figure 2.3.1
When you have set up the MDX resource you can use it in the MDX source. You will also be
able to do a cube query here (MDX query).
Figure 2.3.2

2.4 Excel and OpenOffice Calc sources

To use an Excel workbook or OpenOffice Calc file as a source, you first need to import the
workbook you want to use to the Resources catalogue. You do this by right clicking on a
folder in Resources and selecting Add, Resources, and File. You are then free to browse your
computer and network to import your file.
From the Resources window drag your imported Excel or OpenOffice Calc file to the
Transformer Design window. Transformer will then configure the source for you. You can
still open the source object and make changes to it.
Figure 2.4.1

2.5 CSV source

CSV (Comma Separated Values) source reads any text file where the fields are separated by
commas (default) or another character. You can change the separator to any character. To
use a CSV file as data source, first import it into Resources. When it is located in Resources
you can select it in your CSV source object and specify which character is used as separator
in the CSV file and which rows to use.
Figure 2.5.1

2.6 Surveyor source

Surveyor source is the data source for our survey tool Corporater Surveyor v.1.x. To connect
a Surveyor source to survey results, enter the URL to your Surveyor installation and click List
available surveys. Select the one you want to use from the list.
The URL syntax is: http://<hostname>:<port>/<survey>
Figure 2.6.1

2.7 Surveyor web service source

A Surveyor web service source is the source object that is used to connect to a Corporater Surveyor
v.3.x survey. Enter the path to the Surveyor installation postfixed with
CorpoWebservice/service?wsdl in End point address, the User name and Password for logging on to
Surveyor, and the ID of the Survey.
Figure 2.7.1

2.8 LDAP source

LDAP source is used to load users from Active Directory, or similar software for domain
controllers on other operating systems than Windows. We configure both the connection to
the domain controller and the query to retrieve users or groups from AD in the LDAP source.
Figure 2.7.1
URL: Syntax is:
ldap://<domain controller ip/hostname>:<port>/dc=domainname,dc=domainsuffix.
So if the full URL to the domain controller is Server.Norway.Stavanger.com:389 your URL should then
look like: ldap://Server:398/DC=Norway,DC=Stavanger,DC=com
User and Password: Specify a user that has read rights to the users in the domain. Syntax for user is:
<user> @<domain>. So from the last example your syntax would be:
user@Norway.Stavanger.com

Search filter: Filter your search based on LDAP queries. In this example we use (objectClass=Person),
which will return all the users in the domain. If you are not familiar with LDAP queries it is sufficient
to know only (objectClass=Person) and (objectClass=Group). The rest of the filtering can be done
through Transformer processes.
Fields: Specify which fields from AD you want to include in the source. These will act as columns in
the data stream. Usual fields are:
sAMAccoundName - User ID. This is the username in AD
givenName - First name of the user
sn - Surname of the user
mail - E-mail address
streetAddress - Street address
l (lowercase L) - Location or city
postalCode - Postal code or Zip code
st - State or province
c - Country
telephoneNumber - Telephone number
mobile - Cell phone number

3. Processes in Transformer
Processes are used to select which data to user and manipulate (transform) it before sending it to
the target.
3.1 Aggregation
Aggregation can be used to perform different operations on figures in the source. The
figures can be grouped in different ways before the operations are performed.
The grouping function that is built into EPM data records target performs similar
aggregation on periods (day, month, quarter, year, etc.) as standard, so in many cases it
may be unnecessary to use an Aggregation process.

The columns in the source are listed on the left in Custom tab in the configuration:
Figure 3.1.1
Aggregation function: Choose the type of function you want to use from the drop-
down list:
o SUM - gives the sum of the values
o AVG - calculates the average of the values
o COUNT - counts the number of records
o MIN - gives the lowest value found
o MAX - gives the highest value found
Value column: Select the column in the source that contains the values you want to
use. Only columns defined as numeric will be in the drop-down list.
Describe how to group the entries in the source by adding columns found under
Columns: on the left to Group by: on the right. The entries will be grouped together
according to your definition and figures for each grouping will be treated separately.
Example
The source contains three columns: Org., Dept., and Figures. You want to add the values in
Figures and group by Org and Dept.
In Aggregation function choose "SUM"

In Value column choose "Figures"
In Group by choose first "Org." and then "Dept."
The values in Figures will be added for each Dept. for each Org.
To add all the figures for each Org. select only "Org" in the Group by column.

3.2 Copy
The Copy process in Transformer will clone a data stream to two or more identical streams,
so they can be used in different processes or targets. If we have a source with more than
one value column that is to be used in different targets, we can use a Copy process to clone
the source data to the different targets as shown in figure 3.2.1.
Figure 3.2.1

3.3 Merger
A Merger is used for joining two or more data streams or data sources. The columns in both
must be identical. Duplicates from all the different data streams will always be included in
the merge.
Figure 3.3.1 shows a Merger used to combine two SQL sources.
Figure 3.3.1

3.4 Hierarchy for nodes and organizations

A Hierarchy process object is used for building custom organization or node structures. The
Hierarchy process will therefore always be associated with an Organization target or a Node
target. For this example we will use this data source.
Figure 3.4.1
The structure and logic behind this table is:
parent -> child1-> child11
This means that child11 is a child of child1 which in turn is a child of parent. From this logic
we want to create a sensible organization structure that reflects the parent-child structure
from the data source in the best possible way. In other words, an organization has to be
created as a child of its parent.
Figure 3.4.2

By double clicking on the Hierarchy process object we can configure the hierarchy structure.
In the Hierarchy process object configuration all the available columns are listed under Key
column. We use these columns to build our structure Parent -> child1-> child11 as
mentioned above. We start by moving the column that is to be used as key, in this case
parent, to Corresponding target on the right side. We do this by highlighting parent on the
left side and pressing the arrow pointing to the right.
Figure 3.4.3

After that, highlight parent under Corresponding target and choose the name column in
Name field. This will group together organization ID and organization name. In this example
parent holds the organization ID while the pname holds the organization name. See Figure
3.4.4.
Figure 3.4.4
We then do the same with child1. This will add it as a child of parent. Repeat with child11 it
will become a child of child1. See Figure 3.4.5 and Figure 3.4.6.
Figure 3.4.5 Figure 3.4.6

If we preview the Hierarchy process the data stream should now look like this:
Figure 3.4.7
Use an Organization target when you want to create new organizations in an EPM
database.
3.5 Set constant date

With this Transformer process we can add a static date to the data stream. We can either
overwrite the existing date, if the data stream already has a date column, by choosing a date
column from the drop-down list in Target field, or we can create a new date column that
contains a static date. To create a new column click inside the Target field and write a new
column name. Choose the desired date in Constant date. See figure 3.5.1.
Figure 3.5.1

3.6 Set date

This process sets a new date relative to todays date. We can either overwrite the existing
date, if the data stream already has a date column, or we can create a new date column to
contain a static date. To create a new column click inside the Target field and write a new
column name. See figure 3.6.1.
We adjust the date by entering a time adjustment function in Adjustment function. In Figure
3.6.1 we adjust the date to the end of the month (+EM) and one month ahead (+1M) of
todays date. So if it is July 15, 2011 today the date will be set to August 31, 2011.
Click on the icon to the right of the Adjustment function field to open the Function
calculator.
Figure 3.6.1

3.7 String to date conversion

This process converts dates that are of type string to type date. Dates have to be of type
date in EPM data records target to be accepted.
Figure 3.7.1 shows how to configure the String to date process object. In the Source field we
choose the column that holds the dates as a string. Then we choose (or create) a field which
will hold the converted dates in Target field. In Date format we select the format that
corresponds to the format the date has in Source field. We can also add a function in
Adjustment function if we want to change the dates.
Figure 3.7.1
This process should give us this result:
Figure 3.7.2

3.8 Calculation
The Calculation process is used for doing different calculations on a column or a value in the
data stream. We do the different calculations with the help of different functions.
3.8.1 NUM
NUM is a function that converts the values in a column to numeric values.
The syntax is: NUM(<text>)
where <text> is the column that contains numeric values in text format
Example:
In Figure 3.8.1.1 the column Node ID is a string column that holds values like 500, 501 and
502. We convert the column to numeric format by using NUM and the result is put back in
the same column. This is done by choosing that column in Result Field.
Figure 3.8.1.1

So, if we look at the metadata before and after the conversion it will look like this:
Before:
Figure 3.8.1.2
After:
Figure 3.8.1.3
The column Node ID has now changed from String to Number. NUM can only convert
columns that contain numeric values. It will fail if the column contains string data.
3.8.2 STR
STR is a string calculation. It converts a column to type string very much in the opposite way
that NUM converts a column to numeric.
The syntax is: STR(<number>)
where <number> is a number or the name of a numeric field.

3.8.3 TRIM
The TRIM calculation removes any unwanted whitespace. Lets say that your Organization
IDs in the source are followed by a space (whitespace). You wont be able to match that
Organization ID with any IDs that already exist in EPM as they are not identical. You must
first remove any whitespace by using the TRIM calculation.
The syntax is: TRIM(<text>)
where <text> is a text or a column that contains text.
3.8.4 REPLACE
With REPLACE you can replace a value with another. We can use this, as an example, to
replace a date.
The syntax is: REPLACE(<fieldName>,<toBeReplaced>,<newValue>)
Example
Figure 3.8.4.1
We want to replace October 14, 2011 in Figure 3.8.4.1 with October 24, 2011, i.e. all dates
matching 10.14.2011 will be replaced with 10.24.2011. We accomplish this by doing the
following:

Figure 3.8.4.2
As you can see in Figure 3.8.4.3 the date has now changed:
Figure 3.8.4.3

3.8.5 LEN
If you want to find out how many characters a string consists of you use a LEN (length)
calculation.
The syntax is: LEN(<text>)
where <text> is a text enclosed in double quotation marks, e.g. text, or a field containing a
string.
Please see Figure 3.8.5.1 for configuration and Figure 3.8.7.2 for result.
Figure 3.8.5.1
Figure 3.8.5.2

3.8.6 SUBSTRING
SUBSTRING is used to find and use part of a string. In this example we are going to change
the names in Org.name so that they contain only the first three letters in name shown in
Figure 3.8.4.3.
The syntax is: SUBSTRING(<text>,<Start>,<End>)
where <text> is the column or text to find the substring
We define Start and End by counting the characters in the column or text specified. We start
counting from 0, so the string Norway looks like this:
N O R W A Y
0 1 2 3 4 5
We want to use the first three characters so we set Start to 0 and End to 3. We set End to
the character after the last one we want to use, so by setting End to 3 the search ends after
2. See Figure 3.8.6.1.
Figure 3.8.6.1

Figure 3.8.6.2 shows us the result of this calculation.
Figure 3.8.6.2
All organization names have now been replaced with the first three letters in the original
name.
3.8.7 ABS
ABS (absolute) calculates the absolute value of a number (removing any plus or minus before
the number). In reality, all numbers become positive numbers.
The syntax is: ABS(<number>)
where <number> is a number or the name of the field containing the number.

3.8.8 ROUND
ROUND is used for rounding off a value with decimals to the nearest whole value. For
example 10.25 will be rounded off to 10.
The syntax is: ROUND(<number>)
where <number> is a number or the name of a column that contains numbers.
Example
In this example we will round off the values in the column Decimal in figure 3.8.8.1.
Figure 3.8.8.1
Figure 3.8.8.2 shows that we create a new column named ROUNDED which will hold the
rounded off values, and figure 3.8.8.3 shows the result of this calculation.

Figure 3.8.8.2
If you compare the values in Decimals with the values in ROUNDED you will see that they
have been rounded off correctly.
Figure 3.8.8.3

3.8.9 FIND
FIND is a calculation that is used to look for a substring in string. If the substring is found,
FIND will return the value that represents the index position of the first character of it. If it is
not found, FIND will return -1. See Figure 3.8.9.1 for configuration of the FIND calculation.
The syntax is: FIND(<text>,<search text>)
where <text> is a string enclosed in double quotes, e.g. text or a column that contains a
string, and <search text> is the text we want to find.
Example
In this example we want to find land in the values in the Org.name.
Figure 3.8.9.1
This calculation will give us the index position of the first character in land in the values in
the Org.name column. The result is shown in a new column called FIND. The value -1
indicates that the search text in FIND is not found. See Figure 3.8.9.2.

Figure 3.8.9.2
In this example Norway does not contain land and -1 is therefore set on those rows. On
the rows where the name is England the index is set to 3 because the first character in
land is in position 3:
E N G L A N D
0 1 2 3 4 5 6
For Netherlands the index will be set to 6:
N E T H E R L A N D S
0 1 2 3 4 5 6 7 8 9 10
3.8.10 NOP
NOP calculates the number of periods between two dates.
Choose a field that is to contain the result in the Result Field drop-down list at the top, or
create one by entering a new name.
The syntax is: NOP(<Start date>, <End date>, <Period>)
where <Start date> is the first date in the search interval, <End date> is the last date in the
search interval, and <Period> is the period type to count. <Start date> and <End date> can
contain field names.
Examples
NOP("2010-1-1", "2010-1-31", "D") the number of days between Jan. 1, 2010 and Jan. 31,
2010
NOP("2010-1-10", ${Date},"M") the number of months between Jan. 1, 2010 and the
date contained in the field called 'Date'

3.8.11 DATETONUMBER
DATETONUMBER is a calculator function that converts dates to numbers, i.e. the number of
milliseconds from Jan. 1, 1970. This makes it possible to compare dates.
Choose a field that is to contain the result, or specify a new one, in the Result Field drop-
down list at the top.
The syntax is: DATETONUMBER(<date>)
where <date> is the date to be converted. You can choose a date field or enter a date using
the format "yyyy-mm-dd". Remember to use double quotes when entering a date.
Example 1
DATETONUMBER("2010-03-26")
DATETONUMBER(${Date})
If the date in the field '${Date}' is March 26, 2010 the result of both of the examples above
will be
DATETONUMBER(Date) = 1269558000000
Example 2
Use DATETONUMBER to compare one date with another.
If DATETONUMBER(date1)>DATETONUMBER(date2) means that date1 is after date2

3.9 DePivot
The Transformer DePivot process is used for rotating (pivoting) tables 90 degrees to the
left. The column headers will then become values in a new column that is defined in the
process along with the other existing columns. This is useful in cases where the way the
table is organized makes it is difficult to use the data.
By using a DePivot process, this table:
Header 1 Header 2 Header 3

Value 1 Value 2 Value 3
could become:
New column header 1 New column header 2

Header 1 Value 1
Header 1 Value 2
Header 1 Value 3
Header 2 Value 1
Header 2 Value 2
Header 2 Value 3
Header3 Value 1
Header 3 Value 2
Header 3 Value 3

3.10 Splitter
The Splitter process allows us to split out different data from one data stream to several
data streams. Imagine you are in a room with a group of people and there are three doors
leading out. Above the first door it says blond haired people only, above the second door it
says black haired people only, and above the last door it says the rest.
In the same way we can take one data stream and split it in different directions based on our
criteria. We create doors with criteria/conditions that must be fulfilled to be able to pass
through. We create conditions by opening the Splitter object, right clicking in the grey area,
and selecting Add condition.
Figure 3.10.1
We use calculations to set up our criteria for conditions. You open the calculator by
selecting the Conditions field and pressing the button to the far right.
Example
In this example we want to remove rows where the Organisation ID is 202. We do this by
creating two conditions. See Figure 3.10.2.
Figure 3.10.2
Condition 1 says All rows where Organisation ID is less than or greater than 202
Condition 2 says All rows where Organisation ID = 202

After we have configured our conditions, or doors, we need to define what is to happen
with the different data streams. We do this by dragging lines (connectors) between the
Splitter and the next object. The first line will be from the first condition you created, the
second line will be from the second condition, and so on.
In our example we want to keep only the data stream resulting from Condition 1 and get rid
of the result from Condition 2. So our Transformer project could look something like this:
Figure 3.10.3

3.11 Node and Organization mapping

We can use the Node mapping process to map data from the source to different nodes in
the EPM database.
In this example we want to map all the rows in the source to one node in the database. We
do this by first selecting the source column to use ID (Node ID) in Table column and then
marking all the values in the left column. These are the values that the Node ID column
contains. Then we choose the target node from the EPM database (Node) in the right
column. Press Add to create a new node mapping in the Function field.
Figure 3.11.1
If we preview the data stream now we see that a new column named Node_key has been
created. It defines which node to post the selected data to. This column is to be used in
Node column in an EPM data records target.
The Organization mapping process uses the same procedure but the new column that is
created will have the name Org_Key and it will contain the organization ID the data is
mapped to.

3.12 Multi calculation

A Multi calculation allows us to do multiple selections and calculations in one step. Each
selection is based on a condition that has corresponding operations. We can think of it like:
Condition Operation
IF this is true THEN this happens
AND this happens
ALSO this happens
We can have several conditions with corresponding operations in one Multi Calculation
process. The Multi calculation configuration looks like this:
Figure 3.12.1
We add a new condition by pressing the paper icon in the middle of the object. We create a
new operation by highlighting the condition it is connected to and pressing the paper icon to
the far right.
TIPS: The Condition 1=1 means that the Operation is to be performed on all data records.

4. Targets in Transformer
The Target is where the data is to be written or stored. A project must have at least one target but it
may have more than one.
4.1 CSV target

You can load your data stream to a CSV file if you have one available in Resources. Choose
the file to load the data to and the character to use to separate the values. Your column
headings will then be the first line of text in the CSV file, the first row will be the second line
of text, the second row will become the third line of text, and so on.
Figure 4.1.1

4.2 EPM Data Records target

This target is used for loading data into the EPM database.
Figure 4.2.1
Value column: Select the column that holds the values to be loaded in to the EPM database.
Grouping: Select how the values should be grouped if multiple values have the same
organization, node, node type, and date.
Frequency: Specify which period type to store the data for.
Organization: You can either choose a column that holds a set of organization IDs or you can
manually choose an organization to load the data into.
Node: Select the column that holds the node IDs or manually choose a node to load the data
into.
Node type: Select the column that holds the node type or manually set a node type.
Date: Select the column that holds the dates or manually set a date for the data to be stored
on.
Import: Specify if a new import should be created every time you run the Transformer
project or if the data loaded should overwrite the previous run.

4.3 Excel target

Use this target if you want to load the data to an Excel file located in your Resources. Specify
which Excel workbook to load the data into and which sheet to put it on.
Figure 4.3.1
4.4 Trash can

Trash can is a target that holds all data you do not want to load anywhere. It is often used
with a Splitter process where it acts as a holder for all data you want to remove from a data
stream.

4.5 Node target

A Node target is used for loading nodes and node structures into the EPM database.
Figure 4.5.1
Key: Choose the column that holds the node IDs.
Name: Choose the column that holds the node names.
Description: Choose the column that holds the descriptions, if desired. (Optional)
Attempt best fit: Transformer will automatically try to find the best place to insert when
importing the nodes.
Insert into: Specify when in the node tree the new nodes should be inserted.
Insert as child of column: This is used together with Hierarchy-process to create the desired
tree structure in the database. Choose the column produced by the Hierarchy process. This
column holds a parent key for each node.
If Force structure is marked, the structure in the EPM database will be reorganized to
conform with the structure of the nodes being inserted from the source.
Press Preview and the list of nodes in the right hand pane will be updated with your changes,
so you can check that it is correct. Green indicates nodes that will be inserted, blue denotes
nodes that will be changed, and red nodes that will be moved. This allows you to change the
configuration before the database is actually updated.

4.6 Organization target
Use an Organization target when you want to create or import organizations in an EPM
database. The first three fields on the left (Key, Name, Description) correspond to the EPM
database fields ID, Name, and Description. Key and Name must have content for each
organization. Choose the fields that you want to use as Key, Name, and Description (if any)
from the drop-down lists. The fields you use must be of data type 'String'.
Figure 4.6.1
Key: Choose the column that holds the organization IDs.
Name: Choose the column that holds the organization names.
Description: Choose the column that holds the descriptions (optional)
Attempt best fit: Transformer will automatically try to find the best place to insert
organizations when importing.
Insert into: Specify where in the node tree the new organizations should be inserted.
Insert as child of column: This is used together with a hierarchy process to create the
desired tree structure in the database.
If Force structure is marked, the structure in the EPM database will be reorganized to
conform with the structure of the organizations being inserted from the source.
Press Preview and the list of organizations will be updated in the right hand pane with your
changes, so you can check that it is correct. Green indicates organizations that will be
inserted, blue denotes organizations that will be changed, and red organizations that will be
moved. This allows you to change the configuration before the database is actually updated.

4.7 Group target

Use a Group target for automatic creation of user groups, e.g. from an LDAP source.
Figure 4.7.1
Origin: Choose what origin the groups are produced from: INTERNAL, LDAP, or SSO
Group name column: Choose the column in the source that holds the group names
Primary group token column: Choose the column that holds the primary group tokens
used by AD when the data is imported from LDAP

4.8 User target

Use a User target for automatic creation of users and to load and update users from AD in
the EPM database.
Figure 4.8.1
You need to map the different columns in the source to the different metadata fields in the
system. You do this by choosing a column for each system metadata field.

4.9 Membership target

A Membership target is used to create group memberships for users in the EPM system. The source
data must contain one column for user IDs and one for group IDs, and there must be one row for
each group membership.
Note: This target overwrites existing memberships, so any memberships that users had before this
job is run will be lost. This target changes only the users that have at least one entry in the source
table, so users that are not in the source will not be affected.
Figure 4.9.1
User column: Select the column in the source that contains the user IDs
Group column: Select the column that contains the group IDs

5. Transformer Scheduler
The Scheduler makes it possible to set up when Transformer projects are to be run and plan
automatic runs at regular intervals. You can add different Schedulers to the different
Transformer projects, or you can add a Scheduler for a whole Transformer project group.
The Scheduler configuration determines when and how often a Transformer project or
project group is run. See figure 5.1,
Figure 5.1

6. Validating imported data

After you have imported data to the EPM database by running a Transformer project, you
will be able to view the data in Data records in Configuration Studio. You should validate that
the data loaded are correct.
Note: In Data records you will be able to view all the data imports ever done.
Figure 6.1
In Figure 6.1 we preview the imports for organizations with ID 200, 201, and 202. The filter
located in the window at the bottom right tells us what we are currently previewing. We can
narrow down our scope when validating data by clicking on a node, node type, and/or
organization. We can multi-select by holding down the shift key while clicking.

Figure 6.2
In Figure 6.2 we have narrowed down our search by adding a node, a node type, and an
import to the filter in addition to the organization IDs. So instead of many rows of data we
now only get one.
You clear the filter by clicking on the eraser icon.

7. Exercises
Exercise 1
In the first exercise you are going to create an organization hierarchy using Transformer. You
can find the organizations in the ACCOUNT column in the source file. We only want to
import accounts with 5 digits. The top organization in the hierarchy should have ID = 100 and
the name Corporater Company. On the next level the IDs should be the first digit in the five
digit ACCOUNTs. IDs for the last level should be the five digit ACCOUNTs. You dont need to
think about names in the exercise let the ID and name be the same.
Exercise 2
The next step is to create a node hierarchy using Transformer. The nodes can be found in the
column called CLASS. The top node in the hierarchy should have ID = A, and its name should
be Classes. IDs in the next level should be equal to the first two digits in CLASS. The last level
is the classes. Levels 2 and 3 should use name = ID.
Exercise 3
The last exercise is to import the financial and budget values. This Transformer job should
only have one target. Some financial and budget values are .00 in the source. You need to
replace them with 0. As far as the dates are concerned, it doesnt matter which day of the
month you use, just make sure that the period and year are correct.

Suggested solution Exercise 1: Organization

hierarchy and loading of organizations
We are going to create a hierarchy of organizations using Transformer. We have imported the Excel
file containing the organizations to Resources.
Source
The organizations can be found in the ACCOUNT column.

In Transformer model we create a project group called Loading organizations and nodes. We
create a project called Loading orgs. under it.
Start by dragging the source file from Resources to Transformer design.

Calculation STR
Notice that ACCOUNT is of type number in the source. In order to use it in an Organization target it
must be of type string. So we start by converting ACCOUNT to string by using a Calculation process.
We put the result back into ACCOUNT.

Splitter
The next step is to separate the rows that we dont need from the rows we are going to use, i.e. put
the ACCOUNTS containing 5 digits in a separate data stream. We use a Splitter to do that.
In the first Condition we select the ACCOUNTS with 5 digits by using a LEN function. In the second
Condition we select the ACCOUNTS that dont have 5 digits, I.e length is NOT 5.
Now we need to decide what to do with the different streams.
We will continue to use the first stream and send it to the next process. We dont need the second
stream, so we can send it to a Trash can target.

Multi calculation
Before we can create the organizations in EPM we must give all the ACCOUNTS IDs and names, and
create the hierarchy.
To create the top level organization with ID = 100 and name = Corporater Company we use a Multi
calculation. Because we want to perform these operations on all the rows in the data stream we use
the function 1 = 1 in the Condition. In Operations we create two new columns: TopOrgID with the
value 100 and TopOrgName the value Corporater Company.
Note that 100 and Corporater Company are placed between double quotes (100), indicating that
they are strings.

Calculation SUBSTRING
We use a Calculation process to create the next level. We use the first digit in ACCOUNTS as ID. We
can use a SUBSTRING function to retrieve it and put it into a new field that we call
MiddleLevelIDandName.

Hierarchy
We use a Hierarchy process to define how the organization hierarchy should be set up. First we
select the column from the source that contains the IDs for the top level, TopOrgID, and click the
arrow pointing to the right. Highlight TopOrgID in Corresponding target and find the field that is to be
used as name in Name field.

Similarly, we define the middle level: Select MiddleLevelIDandName in Key column, click the arrow
pointing to the right, and highlight it in Corresponding target. In Name field select
MiddleLevelIDandName, since ID and name should be the same.
Finally, we select ACCOUNTS as both ID and name on the last level. We can see an illustration of the
hierarchy under Corresponding target.

Organization target
Now we have manipulated the data in the source so that it fulfills the requirements and we have
defined the hierarchy to be created. We are ready to create the organizations in the EPM database.
We use an Organization target for that.
We select the column to be used as key in EPM in Key. Note that Key, Name, Description and
Parent_key have been created as a result of what we defined in the Hierarchy process. We can select
them directly in the Organization target. Key = Key, Name = Name. We leave Description empty.
To build the hierarchy in EPM we mark the alternative Insert as child of column and select Parent_Key
in the drop-down. This will create each level under the correct parent level.

Now we can check that the hierarchy will be created correctly by clicking Preview.
The complete project design looks like this:

Suggested solution Exercise 2: Node hierarchy

and loading of nodes
In the next exercise we are going to create a node hierarchy using Transformer. We use the same
source file as in Exercise 1. The nodes can be found in the column called CLASS. The top level in the
hierarchy should have ID = C and be named Classes. The IDs at the next level should be the first two
digits in CLASS. The last level is the classes. In levels 2 and 3 name = ID.
We create the project Loading nodes in the project group Loading organizations and nodes in
Transformer model.
Drag the source file to Transformer design.

Calculation STR
In the source CLASS is data type number, but it must be type string to be used in a Node target. As in
Exercise 2 we use a Calculation process and put the results back into CLASS.

Multi Calculation
We use a Multi Calculation process to create ID = C and name Classes for the top level.

Calculation - SUBSTRING
We use a SUBSTRING function in a Calculation process to create ID and Name for the middle level.
We use the first two digits of CLASS for both of them.

Hierarchy
We define the node hierarchy in a Hierarchy process, in the same way as we defined the organization
hierarchy in Exercise 1. The hierarchy structure is TopNodeID, MiddleLevelIDandName, and CLASS.

Node target
Now the data in the source fulfills the requirements of the Node target and the hierarchy to be
created has been defined. We are ready to load the organizations in the EPM database.
In the Node target we use Key, Name, Description and Parent_key that have been created as a result
of what we defined in the Hierarchy process. Key = Key, Name = Name.
We mark the alternative Insert as child of column and select Parent_Key in the drop-down. This will
create each level under the correct parent level.

Now we can check that the hierarchy will be created correctly by clicking Preview.
The complete project design looks like this:

Suggested solution Exercise 3: Loading financial

and budget values
In the last exercise we are going to import financial and budget values. The exercise contains a
number of prerequisites:
This Transformer job will have only one target
Values that are .00 are to be replaced with 0
It doesnt matter which day of the month you use, just make sure that the period
and year are correct
We will use an EPM Data records target to load the values into the EPM database. To do that we
need the following information: values, organization, node, node type and date, so we need to
process the data before they can be loaded into the database.
Organization: we use ACCOUNT as in Exercise 1
Node: we use CLASS as in Exercise 2
Node type: we need two node types , one for financial values and one for budget values
Date: we use period and year as date so they must be transformed to a format that the
target can handle, and we need to include a day.
Values: we have two value columns in the source AMOUNT_FINANCIAL and
AMOUNT_BUDGET
First we create a new Transformer project group called Loading data with a project called Loading
financial and budget values.
Drag the source from Resources to Transformer design.

Splitter
Now we start creating the project by removing the ACCOUNTS that do not have 5 digits by using a
Splitter, as in Exercise 1. The ones we dont need will be sent to a Trash can target.

Depivot
The structure our source has makes it difficult to fulfill the requirement in the exercise that only one
target is to be used. To fulfill it we need to merge the two value columns to one. We also need to
mark the values so we know which are financial and which are budget. We use a Depivot process for
this. We use it to rotate columns in the source so that the headings are put into a new column and
the content into another.
In our exercise we want to pivot the columns that contain the values ACCOUNT_FINANCIAL and
ACCOUNT_BUDGET. We call the new column containing the values Value and the new column
containing the original headings Node type.
A preview of this process looks like the following screenshot. Now the Value column contains all the
figures and the Node type column contains either ACCOUNT_FINANCIAL or ACCOUNT_BUDGET.

Multi calculation Node type

We can now use the data in the Node type column to create the node types we need. If Node type
contains ACCOUNT_BUDGET it should be replaced with B (budget); if it contains
ACCOUNT_FINANCIAL it should be replaced with A (actual).

Multi calculation Date

Now we need to create a date. According to the exercise text we are to use PERIOD and YEAR as
date. Dates that are entered into EPM must contain day, month and year, so we also need a day.
We choose a Multi calculation process again. The first Condition states that if a PERIOD only has one
digit it should be prefixed with 0 (0 + PERIOD).
The second condition applies to all rows so we use 1=1. A new result field called Date should be
created and contain 01. as day, PERIOD as month, and YEAR as year.

String date conversion

The date we created above is of data type string, so we need to convert it to data type date by using
a String-date conversion process. In addition to changing the data type, the months will be changed
from numbers to text, so 03.01.2010 becomes Mar 1, 2010.

Calculation - REPLACE
The last remaining step is the values to use.
To substitute values that are .00 with 0 we can use a REPLACE function.

Calculation NUM
As you can see in the screenshot below, the data type for Value is string.
It must be converted to data type number before we can load the values into EPM. We use a NUM
function in a Calculation, and we place the results back in the Value-column.

EPM Data records target

Now we are ready to load the data using an EPM Data records target.
In Value column we select Value
In Organization column we select ACCOUNT
In Node column we select CLASS
In Node type column we select Node type
In Date column we select Date
The final project looks like this in Transformer design:

Transformer Advanced

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Transformer Advanced

Uploaded by

Copyright:

Available Formats

Advanced Training, Corporater Transformer

Last updated Dec. 2013 1

Last updated Dec. 2013 2

3.8.4 REPLACE ......................................................................................................................... 32

Last updated Dec. 2013 3

EPM Data model

Period: for what period the data is stored.

Organization: what organization the data is for.

Last updated Dec. 2013 4

Transformer EPM ETL Tool

Last updated Dec. 2013 5

1. Building a Transformer Project

Last updated Dec. 2013 6

1.1.1 Transformer model

1.1.2 Transformer design

Last updated Dec. 2013 7

1.1.5 Preview and progress

Last updated Dec. 2013 8

1.2 Transformer Project Group

1.2.1 Transformer Project

Last updated Dec. 2013 9

2.1 SQL Source

Database: Specify the name of the database to connect to.

Last updated Dec. 2013 10

Last updated Dec. 2013 11

2.2 HTTP source

Last updated Dec. 2013 12

2.3 MDX source

First we need to create an MDX resource to connect to an OLAP database. We do this by

Last updated Dec. 2013 13

2.4 Excel and OpenOffice Calc sources

Last updated Dec. 2013 14

2.5 CSV source

Last updated Dec. 2013 15

2.6 Surveyor source

The URL syntax is: http://<hostname>:<port>/<survey>

Last updated Dec. 2013 16

2.7 Surveyor web service source

Last updated Dec. 2013 17

2.8 LDAP source

URL: Syntax is:

ldap://<domain controller ip/hostname>:<port>/dc=domainname,dc=domainsuffix.

Last updated Dec. 2013 18

sAMAccoundName - User ID. This is the username in AD

givenName - First name of the user

sn - Surname of the user

mail - E-mail address

streetAddress - Street address

l (lowercase L) - Location or city

postalCode - Postal code or Zip code

telephoneNumber - Telephone number

mobile - Cell phone number

Last updated Dec. 2013 19

Last updated Dec. 2013 20

In Aggregation function choose "SUM"

Last updated Dec. 2013 21

Last updated Dec. 2013 22

Figure 3.3.1 shows a Merger used to combine two SQL sources.

Last updated Dec. 2013 23

3.4 Hierarchy for nodes and organizations

The structure and logic behind this table is:

parent -> child1-> child11

Last updated Dec. 2013 24