Using The SCD Type 2 Loader and Lookup Transformations

Using the SCD Type 2 Loader and Lookup
Transformations
Using Type 2 Slowly Changing Dimensions

The SCD Type 2 transformation is used to update dimension tables.
When using this transformation,
• dimension tables can accumulate multiple records per business key
• an alternate column or a combination of columns must be used
as a primary key
• dimension tables are structured to keep historical records
(surrogate/retained key, begin/end date(time)).
38
Copyri g ht © S A S Insti tute Inc. A l l ri g hts reserved.
To support Type 2 slowly changing dimensions, columns are added to a dimension table to
distinguish the current record from historical records such as effective date (or datetime), end date
(or datetime), version number, or current indicator. Other columns are added to provide a primary
key such as a surrogate key or retained key (combined with effective date(time)).
2 Using the SCD Type 2 Loader and Lookup Transformations
Using Type 2 Slowly Changing Dimensions

After the SCD Type 2 transformation is used to update the dimension tables,
the fact table must be updated.
• When a transaction is loaded into the fact table, key values must be
included that link the transaction to the corresponding current record
in each dimension table.
• The historical facts maintain their correct business context to the historical
dimension records.
39
continued...
Updating a Dimension Table
The SCD Type 2 transformation
1. uses the business key to match incoming records to existing records
in a dimension table
Incoming Customer Records
1
Customer Dimension (SK)
40
This Customer Dimension table uses a surrogate key.

Using the SCD Type 2 Loader and Lookup Transformations 3
continued...
2. detects changes to current records
1
Customer Dimension (SK) 2
41
This update occurs on October 27, 2017.

The change in the Country value for customer 65 resulted in a new record for this customer in the
Customer Dimension table. No change is detected for customer 129, so no new record is added for
this customer.
continued...
3. adds records to the dimension table for changes
42
continued...
Updating a Dimension Table: Surrogate Key
4. generates surrogate key values
43
continued...
Updating a Dimension Table: Retained Key
5. generates retained key values and beginning date or datetime values
Customer Dimension (RK) 4
44
Updating a Dimension Table: Surrogate Key

6. updates the current indicator.
5
45
Updating a Dimension Table: Retained Key

7. updates the current indicator.
Customer Dimension (RK) 5
46
continued...
Adding New Records to a Fact Table
The Lookup transformation
1. uses the business key for lookups into the dimension tables
2. uses the current record indicator to determine the current record
Incoming Fact Records
Customer Dimension (SK) 2 1
Fact Table
47
continued...
The Lookup transformation
3. retrieves the primary key values from the dimension tables to be added
as foreign key values to the new fact records.
Fact Table
48
The retrieval of the primary key value from the dimension table maintains referential integrity
between the records in the fact table and the records in the dimension table.

The Table Loader transformation
4. loads the records into the fact table.
4
Fact Table
49
3.02 Multiple Choice Poll

Which transformation is responsible for using the business key and the
current record indicator to retrieve a primary key and add that to an
incoming record before it is loaded into the fact table?
a. SCD Type 1 transformation

b. SCD Type 2 transformation
c. Table Loader transformation
d. Lookup transformation
50
SCD Type 2 Transformation

• applies SCD Type 2 updates to a dimension table
• performs the following tasks:
• detects changes
• tracks changes
• generates key values
• can perform Type 1 updates.
52
The SCD Type 2 transformation can perform Type 1 updates on columns that are not involved in
Type 2 updates. At least one column must be configured for Type 2 updates. If only Type 1 updates
are needed, then use the SCD Type 1 transformation.
Business Key
53
The Business Key tab is used to specify one or more columns in a target dimension table that
represent the business key.
The business key consists of one or more columns that identify a business entity such as a
customer, a product, or an employee.
Change Detection
54
The Detect Changes tab is used to specify one or more columns in a dimension table that are to be
monitored for changes. The business key is used as the basis for change detection. The business
keys in source rows are compared to the business keys in the target.
Change Tracking
55
The Change Tracking tab provides three methods for tracking historical records: (1) begin and end
date (or datetime) values (2) version number (3) current record indicator. Multiple methods can be
selected.
Generated Key
56
The Generated Key tab is used to specify a column to hold the generated key values, as well as an
expression for generating the key values.
By default, the value of the maximum key code is the largest value in the target column that contains
the generated keys. Non-default values are specified by selecting the Define Max Key button on the
Generated Keys tab. In the resulting window, the Maximum Key Code Generator window, you can
obtain an initial value for NewMaxKey from a specified row, column, and table, or you can run a SAS
program to generate that initial value.
To generate a retained key, you select the Generate Retained Key check box in the Generated Keys
tab. With retained keys, source rows that update existing target rows are written into the target with a
copy of the generated key value from the existing row. With surrogate keys, each row in the target
receives a new generated key value.
The generated key is generated at run time for each new row that is added to the target.
Updating Dimension Tables: Closing Out

During the update of a dimensional target, incoming source rows are
compared to the current row for each entry in the target tables.
Depending on data content, source rows can do the following:
begin new entries
57

close out
existing entries
58

add new current
rows to existing
entries
59
An entry consists of all the rows for a business entity represented by a business key value. If a
source row has the same business key and content as a current row in the target, it might indicate
that the entry is being closed out. The entry is closed out if change tracking is implemented with
begin and end datetime values, and if the end datetime value in the source is younger than the same
value in the target. When this is the case, the new end date is written into the target to close out the
entry.
Setup for the Poll

Consider the following image:
60

Given the configuration of the SCD Type 2 transformation, which customer(s)
will be treated as customers with changed data?
a. A customer who has moved to a new country.

b. A customer who has changed their name.
c. Neither customer will be treated as a customer with changed data.
d. Both customers will be treated as customers with changed data.
61
Updating the Fact Table

The fact table in a star schema must also be updated with newly acquired
transactions.
The Lookup transformation is used
to retrieve primary key values from
the dimension tables. These will
be added as the foreign key values
in the fact table. This lookup must
be performed for each row in the source.
The Lookup transformation is used

after the dimension tables have
been updated.
63
The Lookup transformation can be used to load a target table with columns taken from a source and
from a number of lookup tables.
When a job containing a Lookup transformation is run, each source row is processed as follows:
• The key columns in the source row are compared to the key columns in the specified lookup
tables.
• If matches are found, specified lookup values and source values are added to the target row in the
transformation’s target table.
Lookups Tab
64
The Lookups tab in the Lookup transformation is used to specify lookup properties for each lookup
table.
These lookup properties can consist of the following:
• Source to Lookup columns
• Lookup to Target columns
• WHERE expression
• Exceptions
Lookup Properties: Source to Lookup Mapping
65
The Source to Lookup Mapping tab is used to specify one or more columns to be used to find the
matching record in the lookup (dimension) table for each source record.
The business key column (or columns) in the source table is (are) mapped to the corresponding
column (or columns) in the lookup table.
Lookup Properties: Lookup to Target Mapping
66
The Lookup to Target Mapping tab is used to map the one or more column values that are copied
from the lookup (dimension) table to the target (fact) table when a match is found.
The primary key (surrogate key or combination of retained key with BeginDateTime) in the lookup
table is mapped to the corresponding columns in the target table.
Lookup Properties: Selection Criteria
67
The Where tab specifies selection criteria that refine the match between the source row and the
matching rows in the lookup table.
Lookup Properties: Exception Handling
68
The Exceptions tab is used to specify actions to be taken in response to exception conditions
detected in the lookup table.
Available Conditions: Selected Available Actions:

• Lookup table missing • Abort process
• Lookup table contains no records • Add row to error table
• Lookup value not found • Add row to exception table
Error Tables
The Errors tab is used to specify an error table, an exception table, or both.
includes a generated column (source row number) and

any column from the source data
includes four generated columns (exception

information) and any column from the source data
69
Error or exception tables are generated if the appropriate action is selected for the lookup tables.
Setup for the Poll

Consider the following configuration of the Lookup transformation:
72

Which column will be copied to the fact table with the Lookup
transformation?
a. Customer_ID
b. SKCust
c. BeginDateTimeCust
d. EndDateTimeCust
73
Upcoming Demo Steps

1. Review starter objects.
2. Update properties of dimension tables.
3. Configure SCD Type 2 transformations.
4. Specify the processing order.
5. Execute the job and examine the results.
6. Add, configure, and execute Lookup and
Table Loader transformations to load fact table.
7. Change the source reference to a second
month’s data, execute the job, and examine
the results.
8. Change the source reference to a third month’s
data, execute the job, and examine75 the results.
Using the SCD Type 2 Transformation

This demonstration illustrates implementing type 2 updates of dimension tables using the SCD Type
2 transformation.
1. If necessary, access SAS Data Integration Studio using Bruno’s credentials.
a. Select Start  All Programs  SAS  SAS Data Integration Studio.
b. Select My Server as the connection profile.
c. Click OK to close the Connection Profile window. The Log On window appears.
d. Enter Bruno in the User ID field and Student1 in the Password field.
e. Click OK to close the Log On window.
2. Import starter metadata for the job.
a. Click the Folders tab.
b. Right-click the Data Mart Development folder and select Import  SAS Package.
The Import SAS Package Wizard appears.
c. Click Browse next to the Enter the location of the input SAS package file field.
The Browse window appears.
1) If necessary, navigate to D:\Workshop\dift\solutions\ELP_Packages.
Note: This package can also be downloaded from the ELP Packages folder on the
Extended Learning Page.
2) Click DI_ELP_SCDType2Demo.spk.
3) Click OK.
d. Click Next until the Summary screen appears (accept the defaults on each panel).
e. If necessary, create the scdtarget folder.

1) Open Windows Explorer.
2) Navigate to D:\Workshop\dift.
3) Right-click on the Windows Explorer window and select New  Folder.
4) Name the folder scdtarget and press Enter.
5) Return to SAS Data Integration Studio.
f. Click Next on the Summary screen.
g. Click Finish to complete the import.

3. Open the starter job.
b. Expand Data Mart Development  Orion SCD.
c. Verify that the DIFT SCD2
Populate Star Schema job
exists.
4. Right-click DIFT SCD2 Populate

Star Schema and select Open.
In this example, the Splitter transformation is not being used to split rows into different tables but
instead routes columns to different tables. The customer, product, and fact columns all come
from the same source and are directed to the correct table using the Splitter transformation.
5. Examine the source for the job.
a. Right-click table object DIFT SCD2 Source Data and select Properties.
b. Click the Columns tab.
c. Verify that columns 1 to 11 contain customer information.
d. Verify that columns 12 to 19 contain product information.
e. Verify that columns 20 to 29 contain order information.
f. Click Cancel to close the properties window.

6. Investigate the Splitter transformation (named Column Splitter).
a. Right-click the left-most work table under the Splitter transformation and select Properties.
1) Click the Physical Storage tab.
2) Verify that the Physical name field has the value TempForCustDim.
3) Click Cancel to close the properties window.

b. Right-click the middle work table under the Splitter transformation and select Properties.
2) Verify that the Physical name field has the value TempForProdDim.
c. Right-click the right-most work table under the Splitter transformation and select Properties.
2) Verify that the Physical name field has the value TempForOrderFact.
d. Right-click the Splitter transformation and select Properties.
1) Click the Row Selection tab.
a) Verify that TempForCustDim is selected as Target Table.
b) Verify that All Rows will be processed for this target.
c) Click TempForProdDim under Target Table.

d) Verify that All Rows will be processed for this target.
e) Click TempForOrderFact under Target Table.

f) Verify that All Rows will be processed for this target.
2) Click the Mappings tab.

a) On the right side, select TempForCustDim for Target table.
b) Verify that columns 1 to 11 are mapped one-to-one to the 11 target columns.
c) On the right side, select TempForProdDim for Target table.

d) Verify that columns 12 to 19 are mapped one-to-one to the eight target columns.
e) On the right side, select TempForOrderFact for Target table.

f) Verify that columns 20 to 29, plus Customer_ID and Product_ID, are mapped one-
to-one to the 12 target columns.

7. Investigate the Sort transformation named Sort Dedup CustDim.
a. Right-click the Sort transformation and select Properties.
b. Click the Sort By Columns tab.
c. Verify that Customer_ID will be sorted in Ascending sort sequence.
d. Click the Options tab.
e. Verify that SAS Sort is selected in left selection pane.
f. In the main area, verify that Remove duplicate records is set to NODUPKEY.
Note: The first record is the surviving record. Duplicate removal takes place before an SCD
type 2 load because duplicates would lead to many extraneous rows that are closed
out a second after they are recorded.
g. Click Cancel to close the properties window.
8. Investigate the Sort transformation named Sort Dedup ProdDim.
a. Right-click the Sort transformation and select Properties.
b. Click the Sort By Columns tab.
c. Verify Product_ID will be sorted in Ascending sort sequence.
d. Click the Options tab.
e. Verify that SAS Sort is selected in left selection pane.
f. In main area, verify Remove duplicate records is set to NODUPKEY.
g. Click Cancel to close the properties window.
9. Structure the Customer Dimension table for Type 2 slowly changing dimensions.
a. Right-click the DIFT SCD2 Customer Dimension table and select Properties.
c. Click the Customer_ID column to insert new columns after it.
d. Click (New column) four times.
e. Assign the following attributes to the new columns:
Column Name Type Format
SKCust Numeric (None)
BeginDateTimeCust Numeric datetime20.
EndDateTimeCust Numeric datetime20.
CurrRecCust Numeric (None)
f. Click OK to close the DIFT SCD2 Customer Dimension Properties.

10. Structure the Product Dimension table for Type 2 slowly changing dimension.
a. Right-click the DIFT SCD2 Product Dimension table and select Properties.

c. Click the Product_ID column to insert new columns after it.
d. Click (New column) four times.
e. Assign the following attributes to the new columns:
Column Name Type Format
SKProd Numeric (None)
BeginDateTimeProd Numeric datetime20.
EndDateTimeProd Numeric datetime20.
CurrRecProd Numeric (None)
f. Click OK to close the DIFT SCD2 Product Dimension Properties.

11. Configure the SCD Type 2 transformation for the Customer Dimension table.
a. Right-click the transformation named SCD Type 2 CustDim and select Properties.
b. On the General tab, verify that the name of the transformation has been changed to SCD
Type 2 CustDim.
c. Use beginning and end dates to track changes for each customer ID, and use a current
record indicator to keep track of the current record for each customer ID.
1) Click the Change Tracking tab.
2) If necessary, click Use beginning and end dates.
3) Select BeginDateTimeCust as the column name for Beginning Date.
4) Select EndDateTimeCust as the column name for the End Date.
5) Click Use current indicator.

6) Select CurrRecCust as the current indicator column.
The final settings for change tracking should resemble the following:
Note: SAS Data Integration Studio provides default expressions for the Change
Tracking columns. The DATETIME function is used to generate the beginning
datetime value. A datetime constant with a future date is used to specify an open-
ended value for the ending datetime. Custom expressions can be specified if
desired.
Note: Use Version Number is available as an alternative method to track the current
record.
d. Click the Business Key tab.

1) Click New. The DIFT SCD2 Customer Dimension Columns window appears.
2) Click Customer_ID.
3) Click OK to close the DIFT SCD2 Customer Dimension Columns window.
The Customer_ID column appears on the Business Key tab.
e. Click the Generated Key tab.

f. Select SKCust in the Column field.
Note: Default expressions are provided to generate surrogate key values for new and
changed records. To specify a custom expression, click the ellipsis button .
Note: A macro variable, NewMaxKey, is updated automatically with the current maximum
key value. To specify a custom method for obtaining the current maximum key value,
click Define Max Key.
Note: To implement the retained key method, check the Generate retained key option.
g. Click the Detect Changes tab.

h. Double-click the following columns to move from Available to Selected:
Customer_Country
Customer_Name
Customer_Type
Customer_Group
Customer_Age_Group
Note: If no columns are selected on this tab, then all columns will be used to detect
changes, except those used for Change Tracking, Business Key, or Generated Key,
and Type 1 columns.
i. Click the Mappings tab.

j. Click .
k. Verify that all target columns are mapped except SKCust, BeginDateTimeCust,
EndDateTimeCust, and CurrRecCust (these columns will be populated by the
transformation).
’
l. Click OK to close the SCD Type 2 Properties window.
12. Select File  Save to save the job metadata.
13. Configure the SCD Type 2 transformation for the Product dimension.
a. Right-click the SCD Type 2 transformation on the left and select Properties.
b. On the General tab, verify that the name of the transformation has been changed to SCD
Type 2 ProdDim.
c. Use beginning and end dates to track changes for each product ID, and use a current record
indicator to keep track of the current record for each product ID.
2) If necessary, click Use beginning and end dates.
3) Select BeginDateTimeProd as the column name for the beginning date.
4) Select EndDateTimeProd as the column name for the end date.
5) Click Use current indicator.
6) Select CurrRecProd as the value for the current indicator column.
d. Click the Business Key tab.

1) Click New. The DIFT SCD2 Product Dimension Columns window appears.
2) Click Product_ID.
3) Click OK to close the DIFT SCD2 Product Dimension Columns window.
e. Click the Generated Key tab.

f. Select SKProd in the Column field.
g. Click the Detect Changes tab.

h. Double-click the following columns to move from Available to Selected:
Product_Name
Product_Group
Product_Category
Product_Line
i. Click the Mappings tab.

j. Click .
k. Verify that all target columns are mapped except SKProd, BeginDateTimeProd,
EndDateTimeProd, and CurrRecProd. These columns are populated by the transformation.
l. Click OK to close the SCD Type 2 Properties window.

14. Set the control flow order for processing.
a. If necessary, select View  Details to toggle on the Details pane.
b. Click the Control Flow tab.
c. If necessary, reorder the transformations to process in the following order:
15. Select File  Save to save the job metadata to this point.
16. Before running the job, review the source data.

a. Right-click the DIFT SCD2 Source Data object and select Open. The data is displayed in
the View Data window.
There are six records but only four unique Customer_ID values. Notice in particular the
record for Ines Deisser with a Customer_ID value of 65.
b. Scroll right to see the Product_ID column.
All six records represent unique Product _ID values.

c. Select File  Close to close the View Data window.
17. Run the job.
a. Click Run to run the job.
b. Verify that all steps are
completed successfully.
c. If necessary, view the log for
the executed job.
18. View the results.

a. Click the Diagram tab in the job editor window.
b. Right-click the DIFT SCD2 Customer Dimension table and select Open.

d. Right-click the DIFT SCD2 Product Dimension table and select Open.
e. Select File  Close to close the View Data window.


Which of the following statements about the SCD Type 2 transformation
is true?
a. If no columns are selected on the Detect Changes tab, no columns

will be tracked for changes.
b. The SCD Type 2 transformation supports only a surrogate key.
c. Users are unable to change the expression used for the beginning
and end dates on the Change Tracking tab.
d. The SCD Type 2 Transformation can perform Type 1 updates in addition
to Type 2 updates.
78
Using the Lookup Transformation

This demonstration continues from the previous demonstration with steps to add Lookup and Table
Loader transformations to load a fact table. In addition, the source table is changed, the entire job
rerun, and updates to the dimension and fact tables are discussed.
1. If necessary, access SAS Data Integration Studio using Bruno’s credentials.
a. Select Start  All Programs  SAS  SAS Data Integration Studio.
b. Select My Server as the connection profile.
c. Click OK to close the Connection Profile window. The Log On window appears.
d. Enter Bruno in the User ID field and Student1 in the Password field.
e. Click OK to close the Log On window.
2. If necessary, access the job DIFT SCD2 Populate Star Schema.
b. Expand Data Mart Development  Orion SCD.
c. Right-click DIFT SCD2 Populate Star Schema and select Open.
3. Add the Lookup transformation to the process flow.
a. Click the Transformations tab.
b. Expand the Data group.
c. Drag the Lookup transformation to the job editor, and drop it below and to the right of the
two dimension tables.
Drag the Lookup

transformation to the
right and below the two
dimension tables.
4. Specify connections for the Lookup transformation.

a. Verify that three work tables are attached to the Splitter transformation.
Note: By default, the Splitter transformation has two work tables. Additional work tables can
be added by right-clicking the transformation and selecting Add Work Table.
b. Connect the third work table from the Splitter transformation to the first input port of the
Lookup transformation.
Note: The fact table information must connect to the first input port of the Lookup
transformation.
Note: Hold the mouse pointer over the Lookup transformation to reveal the input ports.
c. Connect the DIFT SCD2 Customer Dimension table to the second input port of the Lookup
transformation.
d. Right-click the Lookup transformation and select Ports  Add Input Port.
e. Connect the DIFT SCD2 Product Dimension table to the second input port of the Lookup
transformation.
The job diagram should now resemble the following:
5. Propagate the work table columns to the Lookup result set.

a. Double-click the Lookup transformation to open the Properties.
b. Click the Mappings tab.
c. Click to propagate all the source columns to the target side.
d. Click OK to close the properties window.
6. Add the Table Loader transformation to the process flow.
a. Click the Transformations tab.
b. Expand the Access group.
c. Drag the Table Loader transformation to the job editor and place it below the Lookup
transformation.
d. Connect the Lookup work table to the Table Loader transformation.
7. Add the DIFT SCD2 Order Fact table to the process flow.
b. If necessary, expand Data Mart Development  Orion SCD.
c. Drag the DIFT SCD2 Order Fact table to the job editor and place it below the Table Loader
transformation.
d. Connect the Table Loader transformation to the DIFT SCD2 Order Fact table.
8. Structure the Order Fact table for Type 2 slowly changing dimensions.
a. Right-click the DIFT SCD2 Order Fact table and select Properties.
c. Click the last column to import new columns after it.
d. Click (Import columns). The Import Columns window appears.
1) On the Folders tab, expand Data Mart Development  Orion SCD  DIFT SCD2
Customer Dimension.
2) Double-click SKCust to move it to the selected list.
3) Expand DIFT SCD2 Product Dimension.
4) Double-click SKProd to move it to the selected list.
5) Click OK to close the Import Columns window.

e. Move the SKCust column after the Customer_ID column.
f. Move the SKProd column after the Product_ID column.
g. Click OK to close the DIFT SCD2 Order Fact Properties window.

9. Propagate the new columns back to the Lookup work table.
a. Right-click the Table Loader transformation and select Properties.
b. Click the Mappings tab.
c. Click .
d. Verify that all columns are mapped with one-to-one mappings.
e. On the target table side, click SKCust , hold down the Ctrl key, and click SKProd.
f. Select  Selected Target Columns  To Sources.
g. Verify that the two selected columns are propagated back to the source and mapped.
The resultant Mappings tab should resemble the following:
h. Click OK to close the Table Loader Properties.

10. Configure the Lookup transformation.

a. Right-click the Lookup transformation and select Properties.
b. Click the Lookups tab.
c. Specify lookup mappings for the DIFT SCD2 Customer Dimension table.
1) Click the row for DIFT SCD2 Customer Dimension.
2) Click Lookup Properties.
3) In the Lookup Properties - DIFT SCD2 Customer Dimension window, click the Source to
Lookup Mapping tab.
a) Click the Customer_ID column in the Source table pane.
b) Click the Customer_ID column in the Lookup table pane.
c) Click to map the selected columns.
The Source to Lookup Mapping tab should resemble the following:
The Source to Lookup Mapping specifies the key for the lookup. This should be the
business key.
4) Click the Lookup to Target Mapping tab.

a) Select the SKCust column in the Lookup table pane.
b) Select the SKCust column in the Target table pane.
The Lookup to Target Mapping tab should resemble the following:
The Lookup to Target Mapping specifies the value to be retrieved. This should be the
primary key.
5) Click the Where tab.

a) On the Data Sources tab, expand the SCD2CustDim table.
b) Click the CurrRecCust column.
c) Click Add to Expression to insert the column reference in the Expression Text pane.
d) Type = 1 to complete the expression.
The Where filters the records that are returned from the lookup. We want the current
record for the business key.
6) Click the Exceptions tab.
7) Accept the default combinations of conditions and actions.
Note: Five default exceptions are defined. Two actions both associated with Lookup
Value Not Found can populate an error and an exception table.
8) Click OK to close the Lookup Properties - DIFT SCD2 Customer Dimension window. A
warning appears.
Note: We will visit the Errors tab momentarily.

9) Click OK to close the Warning window.
d. Specify lookup mappings to the DIFT SCD2 Product Dimension table.
1) Click the row for DIFT SCD2 Product Dimension.
3) In the Lookup Properties - DIFT SCD2 Product Dimension window, click the Source to
Lookup Mapping tab.
a) Click the Product_ID column in the Source table pane.
b) Click the Product_ID column in the Lookup table pane.
The Source to Lookup Mapping tab should resemble the following:
The Lookup transformation uses the Product_ID value in the incoming transactions
to do a lookup into the Product Dimension table to find a matching Product_ID
value.
4) Click the Lookup to Target Mapping tab.
a) Click SKProd column in the Lookup table pane.
b) Click SKProd column in the Target table pane.
The Lookup to Target Mapping tab should resemble the following:
The lookup transformation retrieves the SKProd value from the DIFT SCD2 Product
Dimension table and assigns it to the SKProd column in the target table. This links
the transaction in the target table to the current record for the Product ID in the
Product Dimension table.
5) Click the Where tab.
a) On the Data Sources tab, expand the SCD2ProdDim table.
b) Click the CurrRecProd column.
c) Click Add to Expression to insert the column reference in the Expression Text pane.
d) Type = 1 to complete the expression.
The Where tab should resemble the following:
6) Click the Exceptions tab.

7) Accept the default combinations of conditions and actions.
8) Click OK to close the Lookup Properties - DIFT SCD2 Product Dimension window.
e. In the Lookup Properties window, click the Errors tab.
1) Click Create error table.
2) Click Choose columns (under Create error table).

a) Click any column in the Selected columns area.
b) Click to remove all columns from the Selected columns list.
c) Double-click the following items in the Available list to move to the Selected list:
Source Row Number
Order_ID
Customer_ID
Product_ID
The set of columns for the error table should resemble the following:
d) Click OK to close the Choose Error Table Columns window.

3) Click Create exception table.
4) Click Choose columns.

a) Accept the default column selections.
b) Click OK to close the Choose Exception Table Columns window

The Errors tab should now resemble the following:
Recall that the two lookup tables (DIFT SCD2 Customer Dimension and DIFT SCD2
Product Dimension) each had properties set with actions of Add Row To Error Table
and Add Row To Exception Table where each action was associated with the
condition Lookup Value Not Found. These actions require the above configuration on
the Errors tab.
Two additional result sets will now appear for the Lookup transformation, one for the
error table and one for the exception table. These tables can be registered to a
permanent location if desired.
Exception Table
Error Table
f. Click the Mappings tab and review the mappings.
These mappings for the Lookup transformation were established in an earlier step.
g. Click OK to close the Lookup Properties window.
11. Select File  Save to save the job metadata.
12. Configure the Table Loader transformation.
a. Right-click the Table Loader transformation and select Properties.
b. Click the Load Technique tab.
c. Select Append to Existing as the load style.
d. Click OK to close the Table Loader Properties window.

14. Set the control flow order for processing.
a. If necessary, select View  Details to toggle on the Details pane.
b. Click the Control Flow tab.

c. If necessary, reorder the transformations to process in the following order:
1) Column Splitter
2) Sort Dedup CustDim
3) SCD Type 2 CustDim
4) Sort Dedup ProdDim
5) SCD Type 2 ProdDim
6) Lookup
7) Table Loader
16. Verify the order of processing on the Diagram tab in the job editor window.
Recall that the number in the upper left corner of the transformation indicates the order
of processing.
17. Before running the job, review the source data.

a. Right-click the DIFT SCD2 Source Data object and select Open. The data is displayed in
the View Data window.
There are six records but only four unique Customer_ID values. Notice in particular the
record for Ines Deisser with a Customer_ID value of 65.
b. Scroll right to see the Product_ID column.
All six records represent unique Product _ID values.

18. Verify the loaded records in the two dimension tables.
a. Right-click the target table DIFT SCD2 Customer Dimension and select Open.
b. Verify that the four unique records from the source were loaded.

d. Right-click the target table DIFT SCD2 Product Dimension and select Open.
e. Verify that the six unique records from the source were loaded.
f. Select File  Close to close the View Data window.

19. Run the Lookup transformation and the Table Loader transformation.
a. Right-click the Lookup transformation and select Run From Selected Transformation.
b. Verify that the Lookup and Table Loader transformations completed successfully.
20. Review the results.

a. Right-click the DIFT SCD2 Order Fact table and select Open.
b. Scroll right to see the following columns:
The SKCust column has duplicate values for the duplicate values of Customer_ID.
21. Update the job for month 2 (change the source table).
a. Right-click the DIFT SCD2 Source Data object and select Properties.
2) Change the physical name to SCD2SourceDataM02.
b. Click OK to close the DIFT SCD2 Source Data Properties window.
22. View the source data for month 2.
a. Right-click DIFT SCD2 Source Data and select Open.
The data is displayed in the View Data window.
There are five records for month 2. Notice that the name of Ines Deisser (Customer_ID=65)
has changed to Ines Muller.
b. Select File  Close to close the View Data window.
23. Run the job with the new source table.
a. Select Actions  Run.
b. Verify that all steps are completed successfully.

a. Right-click the DIFT SCD2 Customer Dimension table and select Open.
Recall that one of the fields being monitored for changes was the Customer_Name field. For
the records where Customer_ID=65, the name changed, so the original record was marked
inactive (CurrRecCust=0) and the end date was updated on the inactive record. The “new”
record for Customer_ID=65 was given a new surrogate key (SKCust=7) and was marked as
active (CurrRecCust=1).
c. Right-click the DIFT SCD2 Product Dimension table and select Open.
The five new source records represent new Product_ID values, so they were all added to
the product dimension.
d. Select File  Close to close the View Data window.
e. Right-click the DIFT SCD2 Order Fact table and select Open.
All source records from month 1 and month 2 are loaded into the fact table. Notice that both
records for Customer_ID =65 have the correct values for SKCust.
25. Update the job for month 3 (change the source table).
a. Right-click the DIFT SCD2 Source Data object and select Properties.
2) Change the physical name to SCD2SourceDataM03.
b. Click OK to close the DIFT SCD2 Source Data Properties window.
26. View the source data for month 3.

a. Right-click the DIFT SCD2 Source Data table and select Open.
There are 10 records for month 3. Notice that Ines’s (Customer_ID=65) last name changed
back to Deisser.
27. Run the job with the new source table.
a. Select Actions  Run.
b. Verify that all steps are
completed successfully.

a. Right-click the DIFT
SCD2 Customer
Dimension table and
select Open.
Once again, for Customer_ID=65, the Customer_Name changed back to the original name
of Ines Deisser. Thus, the last current record was marked inactive (CurrRecCust=0) and the
end date was updated on the inactive record. The “new” record for Customer_ID=65 was
given a new surrogate key (SKCust=10) and was marked as active (CurrRecCust=1).
In addition, Customer_ID=142 had a name change (Albert to Burt), so the new record is
marked as active, the old record was closed out, and a new surrogate key was generated.
c. Right-click the DIFT SCD2 Product Dimension table and select Open.
Verify that there are 21 records and that all records are “current” (that is, all records have
CurrRecProd=1).
d. Select File  Close to close the View Data window.
e. Right-click the DIFT SCD2 Order Fact table and select Open.
All source records for the three months are loaded into the fact table.
Notice that all records for Customer_ID =65 have the correct values for SKCust.
Similarly, notice that both records for Customer_ID=142 have the correct values for SKCust.
29. Select File  Save to save the diagram and job metadata to this point.
30. Select File  Close to close the job editor window.
Exercises (Optional)
This exercise accesses metadata that are similarly defined to that used in the previous demo.
Properties of the table objects are investigated, properties of the SCD Type 2 and Lookup
transformations are updated, and the job is run using three different source tables.
Use Bruno’s credentials to complete this exercise.
1. Importing Starter Objects
• Access SAS Data Integration Studio using the My Server connection profile using Bruno's
credentials (Bruno/Student1).
• From the Data Mart Development folder, import starter objects from
DI_ELP_SCDType2Exercise.spk (SAS package located in
D:\Workshop\dift\solutions\ELP_Packages).
Note: This package can also be downloaded from the ELP Packages folder on the
Extended Learning Page.
– Choose All Objects.
– If necessary, create the needed physical folder.
Question: How many library objects were imported into the Orion SCD folder?
Answer: _________________________________________________________
Question: What is the name of the physical folder needed for a successful import?
Answer: _________________________________________________________
2. Implementing Type 2 Slowly Changing Dimensions with the Retained Key Method
• If necessary, prepare for the exercise by deleting the physical tables from the
D:\Workshop\dift\scdtarget folder.
• The DIFT SCDR2K Populate Star Schema job is like the job created with surrogate key
columns in the demonstration. Verify that the tables are set up correctly for SCD:
– Verify that the DIFT SCD2RK Customer Dimension table has RKCust,
BeginDateTimeCust, EndDateTimeCust, and CurrRecCust columns following the
Customer_ID column.
– Verify that the DIFT SCD2RK Product Dimension table has RKProd,
BeginDateTimeProd, EndDateTimeProd, and CurrRecProd columns following the
Product_ID column.
• Update the configuration of the SCD Type 2 transformations.
– Edit the properties of the SCD Type 2 CustDim transformation.
– On the Generated Key tab, select the RKCust column. Set the Generate retained key
option.
– Edit the properties of the SCD Type 2 ProdDim transformation.
– On the Generated Key tab, select the RKProd column. Set the Generate retained key
option.
• Update the configuration of the Lookup transformation.
– On the Mappings tab, verify that 12 columns are mapped. The remaining four columns will
be populated by the transformation.
– In the Lookup Properties for the Customer Dimension table, update the Lookup to Target
Mapping to map RKCust and BeginDatetimeCust to the same named columns in the
target.
– Verify that the WHERE expression selects the current record and the Source to Lookup
Mapping tab maps the business key in the dimension table to the lookup table.
– In the Lookup Properties for the Product Dimension table, update the Lookup to Target
Mapping to map RKProd and BeginDatetimeProd to the same named columns in the
target.
– Verify that the WHERE expression selects the current record and the Source to Lookup
Mapping tab maps the business key in the dimension table to the lookup table.
• Run the job for the three months of data provided.
– Verify that the DIFT SCD2 Source Data physical storage references the
SCD2SourceDataM01 table.
– Run the job for month 1.
– Verify that the job completes successfully, view the resultant tables, and verify that the
results make sense.
– Update the DIFT SCD2 Source Data physical storage to reference the
SCD2SourceDataM02 table.
• Run the job for month 2.
• Verify that the job completes successfully, view the resultant tables, and verify that the results
make sense.
• Update the DIFT SCD2 Source Data physical storage to reference the SCD2SourceDataM03
table.
• Run the job for month 3.
• Verify that the job completes successfully, view the resultant tables, and verify that the results
make sense.
• Save and close the job.
Solutions
Solutions to Exercises
1. Implementing Type 2 Slowly Changing Dimensions with the Retained Key Method
a. If necessary, access SAS Data Integration Studio using Bruno’s credentials.
1) Select Start  All Programs  SAS  SAS Data Integration Studio.
2) Select My Server as the connection profile.
3) Click OK to close the Connection Profile window. The Log On window appears.
1) Enter Bruno in the User ID field and Student1 in the Password field.
2) Click OK to close the Log On window.
Import a copy of the job created in the SCD Type 2 demonstration modified such that the
dimension tables can be set up with a retained key rather than a surrogate key.
b. Import starter objects.
1) Click the Folders tab.
2) Right-click the Data Mart Development folder and select Import  SAS Package.
3) Select the Browse button.
4) Navigate to D:\Workshop\dift\solutions\ELP_Packages.
5) Select the DI_ELP_SCDType2Exercise.spk package and press OK.
6) Accept the default selections for the rest of the pages in the wizard by clicking Next.
7) Click Finish.
c. Verify that the imported tables are correctly configured for SCD.
1) Right-click the DIFT SCD2RK Customer Dimension table and select Properties.
2) Click the Columns tab.
3) Verify that RKCust, BeginDateTimeCust, EndDateTimeCust, and CurrRecCust

columns follow the Customer_ID column.

5) Verify that the physical name of the table is SCD2RKCustDim.
6) Click Cancel to close the DIFT SCD2RK Customer Dimension Properties window.
7) Right-click the DIFT SCD2RK Product Dimension table and select Properties.
8) Click the Columns tab.
9) Verify that RKProd, BeginDateTimeProd, EndDateTimeProd, and CurrRecProd

columns follow the Product_ID column.

11) Verify that the physical name of the table is SCD2RKProdDim.
12) Click Cancel to close the DIFT SCD2RK Product Dimension Properties window.
d. Open the imported job.

1) Double-click the job named DIFT SCD2RK Populate Star Schema.
2) Verify that the job flow diagram resembles the following:
e. Update the configuration of the SCD Type 2 CustDim transformation.

1) Right-click the SCD Type 2 CustDim transformation and select Properties.
2) Click the Generated Key tab.
3) Select RKCust as the column for the generated key.
4) Click the Generate retained key option.

6) Verify that changes will be tracked using begin and end date fields, as well as a field to
indicate the current record.
7) Click the Business Key tab.
8) Verify that Customer_ID is defined as the business key.
9) Click the Detect Changes tab.
10) Verify that changes will be detected for five fields.
11) Click OK to close the SCD Type 2 CustDim Properties.
f. Update the configuration of the SCD Type 2 ProdDim transformation.
1) Right-click the SCD Type 2 ProdDim transformation and select Properties.
2) Click the Generated Key tab.
3) Select RKProd as the column for the generated key.
4) Click the Generate retained key option.
6) Verify that changes will be tracked using begin and end date fields, as well as a field to
indicate the current record.
7) Click the Business Key tab.
8) Verify that Product_ID is defined as the business key.
9) Click the Detect Changes tab.
10) Verify that changes will be detected for four fields.
11) Click OK to close the SCD Type 2 ProdDim Properties.
g. Update the configuration of the Lookup transformation.
1) Right-click the Lookup transformation and select Properties.
2) Click the Mappings tab.
3) Verify that 12 columns are mapped.
Note: The remaining fields will be populated by the transformation.
4) Click the Lookups tab.

5) Click DIFT SCD2RK Customer Dimension.
The Lookup Properties - DIFT SCD2RK Customer Dimension window appears.
a) On the Source to Lookup Mapping tab, verify the mapping from Customer_ID in the
source table to Customer_ID in the lookup table.
b) Click the Lookup to Target Mapping tab.
c) Click to map RKCust and BeginDateTimeCust from the lookup table to the target
table.
d) Click the Where tab.

e) Verify the expression text.
f) Click OK to close the Lookup Properties - DIFT SCD2RK Customer Dimension

window.
7) On the Lookups tab, click DIFT SCD2RK Product Dimension.

The Lookup Properties - DIFT SCD2RK Product Dimension window appears.
a) On the Source to Lookup Mapping tab, verify the mapping from Product_ID in the
Source table to Product _ID in the Lookup table.
b) Click the Lookup to Target Mapping tab.
c) Click to map RKCust and BeginDateTimeCust from the lookup table to the target
table.
d) Click the Where tab.

e) Verify the expression text.
f) Click OK to close the Lookup Properties - DIFT SCD2RK Product Dimension window.
9) Click the Errors tab.
10) Verify that both the Error and Exception tables are configured.
11) Click OK to close the Lookup Properties window.
h. Verify that the source data are for month 1.
1) In the diagram, right-click the DIFT SCD2 Source Data table and select Properties.
3) Verify that the physical name is SCD2SourceDataM01.
4) Click OK to close the DIFT SCD2 Source Data Properties window.
i. Select File  Save to save the job metadata.
j. Run the job for the first month.
1) Click Run.
2) Verify that the job completed successfully.
k. View the results.
1) Right-click the DIFT SCD2RK Customer Dimension table and select Open. The data is
displayed in the View Data window.
2) Select File  Close to close the View Data window.

3) Right-click the DIFT SCD2RK Product Dimension table and select Open.

5) Right-click the DIFT SCD2RK Order Fact table and select Open.

l. Review the data and run the job for the second month.
1) In the diagram, right-click the DIFT SCD2 Source Data table and select Properties.
a) Click the Physical Storage tab.
b) Update the physical name to SCD2SourceDataM02.
c) Click OK to close the DIFT SCD2 Source Data Properties window.
2) Right-click the DIFT SCD2 Source Data table and select Open.
There are five records for month 2. Notice that the name of Ines Deisser (Customer_ID
=65) has changed to Ines Muller, and the name field is a field being tracked for changes.
m. Select File  Save to save the job metadata.
n. Run the job for the first month.
1) Click Run.
o. View the results.
1) Right-click the DIFT SCD2RK Customer Dimension table and select Open.
The name change resulted in a new record for Customer_ID =65. The new record has
retained the value of the generated key, RKCust. The end datetime and current record
indicator values in the previous record were updated.
All five source records represent new Product_ID values, so they were all added to
the product dimension with new generated key values.
All source records from month 1 and month 2 are loaded into the Fact table. Notice
that both records for Customer_ID=65 have the correct value for RKCust and the
correct values for BeginDateTimeCust. Because RKCust is not unique in either the
Fact table or the Dimension table, the combination of RKCust and
BeginDateTimeCust must be used as the key to relate a record in the Fact table to
the corresponding record in the Dimension table.
p. Review the data and run the job for the third month.
1) From the job diagram, right-click the DIFT SCD2 Source Data table and select
Properties.
3) Update the physical name to SCD2SourceDataM03.
4) Click OK to close the DIFT SCD2 Source Data Properties window.
5) Right-click the DIFT SCD2 Source Data table and select Open.
There are 10 records for month 3. Notice that Ines’ (Customer_ID=65) last name
changed back to Deisser. In addition, Customer_ID=142 has a changed first name.
q. Select File  Save to save the job metadata.
r. Run the job for the first month.
1) Click Run.
s. View the results.
1) Right-click the DIFT SCD2RK Customer Dimension table and select Open.
Again, the name change resulted in a new (third) record for Customer_ID=65 with a
retained generated key value (RKCust). The end datetime and current record indicator in
the previous old record were updated.
Customer_ID=142 also had a name change, which resulted in a new record.
4) Verify that no changed data occurred for this dimension.
All 21 source records from months 1, 2, and 3 are loaded into the Fact table. Notice that
all records for Customer_ID=65 have the correct value for RKCust as well as that for
BeginDateTimeCust. The combination of RKCust and BeginDateTimeCust represents
the (foreign) key in the Fact table that links to the corresponding (primary) key in the
dimension table.
8) Select File  Close to close the job.
Solutions to Student Activities (Polls/Quizzes)
3.01 Multiple Choice Poll – Correct Answer

Given that the business key is contained in the Customer_ID column, does
the Key column contain a retained key or a surrogate key?
The value of Key is different for the same values of
a. Retained key Customer_ID, indicating that it is a surrogate key.
b. Surrogate key
c. There is not enough
information to decide.
d. It contains neither a
surrogate nor a retained key.
34

Which transformation is responsible for using the business key and the
current record indicator to retrieve a primary key and add that to an
incoming record before it is loaded into the fact table?
a. SCD Type 1 transformation

b. SCD Type 2 transformation
c. Table Loader transformation
d. Lookup transformation
51

Given the configuration of the SCD Type 2 transformation, which customer(s)
will be treated as customers with changed data?
a. A customer who has moved to a new country.

b. A customer who has changed their name.
c. Neither customer will be treated as a customer with changed data.
d. Both customers will be treated as customers with changed data.
The customer in (B) changed their name, and the SCD Type 2 transformation was
configured to track changes on that column, so they will be treated as a customer with
changed data. The Customer_Country variable was not used to detect changes, so the
customer in (A) will not be treated as a customer with changed data.
62

Which column will be copied to the fact table with the Lookup
transformation?
a. Customer_ID
b. SKCust
c. BeginDateTimeCust
d. EndDateTimeCust
The SKCust column is the primary key for the Customer Dimension table and
is copied to the Order Fact table on the Lookup to Target Mapping tab of the
Lookup transformation.
74

Which of the following statements about the SCD Type 2 transformation
is true?
a. If no columns are selected on the Detect Changes tab, no columns will

be tracked for changes.
b. The SCD Type 2 transformation supports only a surrogate key.
c. Users are unable to change the expression used for the beginning
and end dates on the Change Tracking tab.
d. The SCD Type 2 Transformation can perform Type 1 updates in addition
to Type 2 updates.
79
3.06 Poll – Correct Answer

Can SCD Type 1 updates be performed by transformations other than
the SCD Type 1 transformation?
 True
 False
The SCD Type 2 transformation can perform SCD Type 1 updates if at least one
column has been selected for SCD Type 2 updates.
103

Using The SCD Type 2 Loader and Lookup Transformations

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Using The SCD Type 2 Loader and Lookup Transformations

Uploaded by

Copyright:

Available Formats

Using the SCD Type 2 Loader and Lookup

Using Type 2 Slowly Changing Dimensions

Using Type 2 Slowly Changing Dimensions

This Customer Dimension table uses a surrogate key.

Incoming Customer Records

This update occurs on October 27, 2017.

Customer Dimension (SK) 3

Customer Dimension (SK) 4

Customer Dimension (RK) 4

Updating a Dimension Table: Surrogate Key

Updating a Dimension Table: Retained Key

Customer Dimension (RK) 5

Customer Dimension (SK) 2 1

Customer Dimension (SK)

Adding New Records to a Fact Table

3.02 Multiple Choice Poll

a. SCD Type 1 transformation

SCD Type 2 Transformation

Updating Dimension Tables: Closing Out

begin new entries

Updating Dimension Tables: Closing Out

Updating Dimension Tables: Closing Out

Setup for the Poll

3.03 Multiple Choice Poll

a. A customer who has moved to a new country.

Updating the Fact Table

The Lookup transformation is used

Lookup Properties: Source to Lookup Mapping

Lookup Properties: Lookup to Target Mapping

Lookup Properties: Selection Criteria

Lookup Properties: Exception Handling

Available Conditions: Selected Available Actions:

includes a generated column (source row number) and

includes four generated columns (exception

Setup for the Poll

3.04 Multiple Choice Poll

Upcoming Demo Steps

Using the SCD Type 2 Transformation

e. If necessary, create the scdtarget folder.

g. Click Finish to complete the import.

4. Right-click DIFT SCD2 Populate

c. Verify that columns 1 to 11 contain customer information.

d. Verify that columns 12 to 19 contain product information.

e. Verify that columns 20 to 29 contain order information.

f. Click Cancel to close the properties window.

3) Click Cancel to close the properties window.

c) Click TempForProdDim under Target Table.

e) Click TempForOrderFact under Target Table.

2) Click the Mappings tab.

c) On the right side, select TempForProdDim for Target table.

e) On the right side, select TempForOrderFact for Target table.

3) Click Cancel to close the properties window.

Column Name Type Format

SKCust Numeric (None)

BeginDateTimeCust Numeric datetime20.

EndDateTimeCust Numeric datetime20.

CurrRecCust Numeric (None)

f. Click OK to close the DIFT SCD2 Customer Dimension Properties.

b. Click the Columns tab.

Column Name Type Format

SKProd Numeric (None)

BeginDateTimeProd Numeric datetime20.

EndDateTimeProd Numeric datetime20.