Using dynamic lookup cache

Applies to:
Informatica PowerCenter

Summary
This article outlines the possibility to eliminate duplicate records by using a dynamic lookup cache.

Author Bio
Author(s): Matthias Urech Company: interface-development.com Created on: August 21, 2009 Matthias Urech, founder and project leader of interface - development.com, has a proven track record of applying data integration solutions for several companies across the industry. He played a key role to implement projects in the area of data integration, data migration, data consolidation and data warehousing. Matthias contributed several articles about data integration best practices for Informatica Technology Network, resulting in nomination as member of developer wall of fame and receiving an award as contributor of the year in 2008.

Table of Contents
Introduction .........................................................................................................................................................2 Analyzing the problem ........................................................................................................................................2 Walkthrough: Using dynamic lookup cache........................................................................................................3
Step 1: Creating Source and Source Qualifier .............................................................................................................3 Step 2: Creating Target................................................................................................................................................4 Step 3: Filter inactive records.......................................................................................................................................4 Step 4: Sorting records for latest change.....................................................................................................................5 Step 5: Lookup target data...........................................................................................................................................6 Step 6: Flagging data for insert or update....................................................................................................................7 Step 7: Creating Workflow and Session.......................................................................................................................8

Disclaimer and Liability notice ............................................................................................................................9

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 1

Using dynamic lookup cache

Introduction
This article will dive into the deep blue of using a dynamic lookup cache that inserts and updates rows in the cache as it passes rows to the target. The example used in this article is dealing with the issue that a status is stored multiple times in the source system without flagging the actual record. We will analyze the problem and go step by step in building a solution by using dynamic lookup cache and storing only the actual status in the target.

Analyzing the problem

Figure 1 shows example table OBJECT_STATUS. We want to get the actual STATUS_ID for each OBJECT_ID. By taking a closer look at the table we already recognize a pattern: Each status has a date stamp Equal date stamps contain a sequential number (attribute CHANGE_NO) An inactive status is marked

Figure 1: source table OBJECT_STATUS

Given the fact that we know the pattern, we are able to roughly draw the solution: Sort data by attribute OBJECT_ID, CHANGE_DATE, CHANGE_NO Use dynamic lookup cache to determine if current records exists in target Flag record to force insert or update in target

Once the solution has been built and ran, the yellow marked records in Figure 1 are reflecting the result (= unique value for each OBJECT_ID)

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 2

Using dynamic lookup cache

Walkthrough: Using dynamic lookup cache

In this section you will accomplish the following activities: 1. Creating Source and Source Qualifier 2. Creating Target 3. Filter inactive records 4. Sorting records for latest change 5. Lookup target data 6. Flagging data for insert or update 7. Creating Workflow and Session Figure 2 illustrates the result of our activities.

Figure 2: mapping m_USING_DYNAMIC_LOOKUP_CACHE

Step 1: Creating Source and Source Qualifier To create Source and Source Qualifier: 1. Create a new mapping m_USING_DYNAMIC_LOOKUP_CACHE 2. Create a Source by defining the table definition. The source table has the following data definitions:

Figure 3: data definition of Source

3. Add Source and Source Qualifier to your mapping and connect the ports

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 3

Using dynamic lookup cache

Step 2: Creating Target To create a target: 1. Create a relational Target to stage the output. The Target has the following data definitions:

Figure 4: data definition of Target

2. Create physical target in your preferred database by running the create table statement 3. Add the Target to your mapping Step 3: Filter inactive records To filter inactive records: 1. Add a Filter transformation to your mapping 2. Select all ports of the Source Qualifier and drop them into the Filter transformation 3. Edit the transformation and specify the values in tab Properties as follows:

Figure 5: settings of Filter transformation

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 4

Using dynamic lookup cache

Step 4: Sorting records for latest change To sort records for latest change: 1. Add as Sorter transformation to your mapping 2. Select all ports of the Filter and drop them into the Sorter transformation 3. Edit the transformation, select checkbox Key and specify Ascending as sort order for the following ports: OBJECT_ID, CHANGE_DATE, CHANGE_NO 4. Specify other settings according to figure 6:

Figure 6: settings of Sorter transformation

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 5

Using dynamic lookup cache

Step 5: Lookup target data To lookup target data: 1. Add a Lookup Transformation to your mapping. Select target OBJECT_STATUS in the popup window and click [OK] button 2. Select all ports from the Sorter and drop it to the Lookup transformation 3. Edit the transformation and select the Condition tab. Add a new condition and specify OBJECT_ID as port to be matched 4. Select the Properties tab and select checkbox Dynamic Lookup Cache and Insert Else Update 5. Associate lookup ports with input ports 6. Enter other settings according to figure 7:

Figure 7: settings of Lookup transformation

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 6

Using dynamic lookup cache

Step 6: Flagging data for insert or update To flag data for insert or update: 1. 2. 3. 4. 5. Add an Update Strategy transformation to your mapping Select port NewLookupRow of the Lookup and drop it to the Update Strategy transformation Select all ports of the Sorter and drop them to the Update Strategy transformation Edit the Update Strategy transformation and select tab Properties Enter the following value in the Update Strategy Expression:
IIF(NewLookupRow = 1, DD_INSERT, DD_UPDATE)

Note: The Informatica Server assigns a value to the port, depending on the action it performs to the lookup cache: NewLookupRow Value 0 1 2
Table 1: NewLookupRow values

Description The Informatica Server inserts the row into the cache. The Informatica Server inserts the row into the cache. The Informatica Server updates the row in the cache.

6. Select all ports of the Update Strategy transformation and connect them to the Target 7. Save your mapping m_USING_DYNAMIC_LOOKUP_CACHE

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 7

Using dynamic lookup cache

Step 7: Creating Workflow and Session To create Workflow and Session: Create a workflow wf_USING_DYNAMIC_LOOKUP_CACHE Create a session for mapping m_USING_DYNAMIC_LOOKUP_CACHE Select tab Sources and specify the source directory and file name (object_status.dat) Select tab Targets, specify the database to connect and select Insert, Update Else Insert and Truncate action in the Property section 5. Perform all other required settings 6. Save the workflow wf_USING_DYNAMIC_LOOKUP_CACHE 1. 2. 3. 4.

Figure 8: workflow wf_USING_DYNAMIC_LOOKUP_CACHE

7. Run the workflow and see the output:

Figure 9: output of session workflow wf_USING_DYNAMIC_LOOKUP_CACHE

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 8

Using dynamic lookup cache

Disclaimer and Liability notice

Informatica offers no guarantees and assumes no responsibility or liability of any type with respect to the content of this software asset, including any liability resulting from incompatibility between the content within this asset and the materials and services offered by Informatica. You agree that you will not hold, or seek to hold, Informatica responsible or liable with respect to the content of this software asset.

Informatica Technology Network 2009 Informatica Corporation. All Rights Reserved.

http://technet.informatica.com 9