Wednesday, 26 November 2014

Load Last N Records of File into Target Table - Informatica

How to load only the last N rows from source file into the target table using the mapping in informatica?
First take a look at the below data in the source file:
Products
--------
Windows
Linux
Unix
Ubuntu
Fedora
Centos
Debian
I want to load only the last record or footer into the target table. The target should contain only the product "Debain". Follow the below steps for implementing the mapping logic in informatica:
  • The mapping flow and the transformations are shown below:
SRC->SQ->EXPRESSION->SORTER->EXPRESSION->FILTER->TGT
  • Create a new mapping and drag the source into the mapping. By default, it creates the source qualifier transformation.
  • Now create an expression transformation and drag the ports from source qualifier into the expression transformation. In the expression transformation, create the below additional ports and assign the corresponding expressions:
v_count (variable port) = v_count+1
o_count (output port) = v_count
  • The output of expression transformation is
Products, o_count
-----------------
Windows, 1
Linux,   2
Unix,    3
Ubuntu,  4
Fedora,  5
Centos,  6
Debian,  7
  • Now connect the expression transformation to a sorter transformation and sort the rows on the o_count port in descending order. The output of sorter transformation is shown below:
Products
--------
Debian
Centos
Fedora
Ubuntu
Unix
Linux
Windows
  • Create another expression transformation and connect the Products port of sorter to expression transformation. Create the following ports in the expression transformation:
v_count (variable port) = v_count+1
o_count (output port) = v_count
  • Connect the expression to a filter transformation and specify the filter condition as o_count = 1.
  • Connect the filter to the target and save the mapping.

Informatica online training - Informatica Jobs

Load Alternative Records / Rows into Multiple Targets - Informatica

How to load records alternativly into multiple targets in informatica? Implement mapping logic for this.
I have a source file which contains N number of records. I want to load the source records into two targets, such that first row goest into target 1, second row goes into target2, third row goes into target3 and so on.
Let see how to create a mapping logic for this in informatica with an example. Consider the following source flat file as an example:
Products
---------
Informatica
Datastage
Pentaho
MSBI
Oracle
Mysql
The data in the targets should be:
Target1
-------
Informatica
Pentaho
Oracle

Target2
-------
Datastage
MSBI
Mysql
Solution:
The mapping flow and the transformations used are mentioned below:
SRC->SQ->EXP->RTR->TGTS
  • First create a new mapping and drag the source into the mapping.
  • Create an expression transformation. Drag the ports of source qualifier into the expression transformation. Create the following additional ports and assign the corresponding expressions:
v_count (variable port) = v_count+1
o_count (output port) = v_count
  • Create a router transformation and drag the ports (products, v_count) from expression transformation into the router transformation. Create an output group in the router transformation and specify the following filter condition:
MOD(o_count,2) = 1
  • Now connect the output group of the router transformation to the target1 and default group to target2. Save the mapping.
In the above solution, I have used expression transfromation for generating numbers. You can also use sequence generator transformation for producing sequence values.
Informatica online training - Informatica Jobs

Union Transformation in Informatica

Union transformation is an active and connected transformation. It is multi input group transformation used to merge the data from multiple pipelines into a single pipeline. Basically it merges data from multiples sources just like the UNION ALL set operator in SQL. The union transformation does not remove any duplicate rows.
Union Transformation Guidelines
The following rules and guidelines should be used when using a union transformation in a mapping
  • Union transformation contains only one output group and can have multiple input groups.
  • The input groups and output groups should have matching ports. The datatype, precision and scale must be same.
  • Union transformation does not remove duplicates. To remove the duplicate rows use sorter transformation with "select distinct" option after the union transformation.
  • The union transformation does not generate transactions.
  • You cannot connect a sequence generator transformation to the union transformation.
  • Union transformation does not generate transactions.
Creating union transformation
Follow the below steps to create a union transformation
  1. Go the mapping designer, create a new mapping or open an existing mapping
  2. Go to the toolbar-> click on Transformations->Create
  3. Select the union transformation and enter the name. Now click on Done and then click on OK.
  4. Go to the Groups Tab and then add a group for each source you want to merge.
  5. Go to the Group Ports Tab and add the ports.
Components of union transformation
Configure the following tabs of union transformation
  • Transformation: You can enter name and description of the transformation
  • Properties: Specify the amount of tracing level to be tracked in the session log.
  • Groups Tab: You can create new input groups or delete existing input groups.
  • Group Ports Tab: You can create and delete ports for the input groups.
Note: The ports tab displays the groups and ports you create. You cannot edit the port or group information in the ports tab. To do changes use the groups tab and group ports tab.
Why union transformation is active
Union is an active transformation because it combines two or more data streams into one. Though the total number of rows passing into the Union is the same as the total number of rows passing out of it, and the sequence of rows from any given input stream is preserved in the output, the positions of the rows are not preserved, i.e. row number 1 from input stream 1 might not be row number 1 in the output stream. Union does not even guarantee that the output is repeatable.
Union Transformation Example
1. There are two tables in the source. The table names are employees_US and employees_UK and have the structure. Create a mapping to load the data of these two tables into single target table employees?
Informatica online training - Informatica Jobs

Update Strategy Transformation in Informatica

Update strategy transformation is an active and connected transformation. Update strategy transformation is used to insert, update, and delete records in the target table. It can also reject the records without reaching the target table. When you design a target table, you need to decide what data should be stored in the target.
When you want to maintain a history or source in the target table, then for every change in the source record you want to insert a new record in the target table.
When you want an exact copy of source data to be maintained in the target table, then if the source data changes you have to update the corresponding records in the target.
The design of the target table decides how to handle the changes to existing rows. In the informatica, you can set the update strategy at two different levels:
  • Session Level: Configuring at session level instructs the integration service to either treat all rows in the same way (Insert or update or delete) or use instructions coded in the session mapping to flag for different database operations.
  • Mapping Level: Use update strategy transformation to flag rows for inert, update, delete or reject.
Flagging Rows in Mapping with Update Strategy:
You have to flag each row for inserting, updating, deleting or rejecting. The constants and their numeric equivalents for each database operation are listed below.
  • DD_INSERT: Numeric value is 0. Used for flagging the row as Insert.
  • DD_UPDATE: Numeric value is 1. Used for flagging the row as Update.
  • DD_DELETE: Numeric value is 2. Used for flagging the row as Delete.
  • DD_REJECT: Numeric value is 3. Used for flagging the row as Reject.
The integration service treats any other numeric value as an insert.
Update Strategy Expression:
You have to flag rows by assigning the constant numeric values using the update strategy expression. The update strategy expression property is available in the properties tab of the update strategy transformation.
Each row is tested against the condition specified in the update strategy expression and a constant value is assigned to it. A sample expression is show below:
IIF(department_id=10, DD_UPDATE, DD_INSERT)
Mostly IIF and DECODE functions are used to test for a condition in update strategy transformation.
Update Strategy and Lookup Transformations:
Update strategy transformation is used mostly with lookup transformation. The row from the source qualifier is compared with row from lookup transformation to determine whether it is already exists or a new record. Based on this comparison, the row is flagged to insert or update using the update strategy transformation.
Update Strategy and Aggregator Transformations:
If you place an update strategy before an aggregator transformation, the way the aggregator transformation performs aggregate calculations depends on the flagging of the row. For example, if you flag a row for delete and then later use the row to calculate the sum, then the integration service subtracts the value appearing in this row. If it’s flagged for insert, then the aggregator adds its value to the sum.
Important Note:
Update strategy works only when we have a primary key on the target table. If there is no primary key available on the target table, then you have to specify a primary key in the target definition in the mapping for update strategy transformation to work.
Informatica online training - Informatica Jobs

Transaction Control Transformation in Informatica

Transaction Control is an active and connected transformation. The transaction control transformation is used to control the commit and rollback of transactions. You can define a transaction based on varying number of input rows. As an example, you can define a transaction on a group rows in the employees data using the department Id as a key.
In the informatica power center, you can define the transaction at the following levels:
  • Mapping level: Use the transaction control transformation to define the transactions.
  • Session level: You can specify the "Commit Type" option in the session properties tab. The different options of "Commit Type" are Target, Source and User Defined. If you have used the transaction control transformation in the mapping, then the "Commit Type" will always be "User Defined"
When you run a session, the integration service evaluates the expression for each row in the transaction control transformation. When it evaluates the expression as commit, then it commits all the rows in the transaction to the target(s). When the integration service evaluates the expression as rollback, then it roll back all the rows in the transaction from the target(s).
When you have flat file as the target, then the integration service creates an output file for each time it commits the transaction. You can dynamically name the target flat files. Look at the example for creating flat files dynamically - Dynamic flat file creation.
Creating Transaction Control Transformation
Follow the below steps to create transaction control transformation:
  • Go to the mapping designer, click on transformation in the toolbar, Create.
  • Select the transaction control transformation, enter the name and click on Create and then Done.
  • You can drag the ports in to the transaction control transformation or you can create the ports manually in the ports tab.
  • Go to the properties tab. Enter the transaction control expression in the Transaction Control Condition.
Configuring Transaction Control Transformation
You can configure the following components in the transaction control transformation:
  • Transformation Tab: You can rename the transformation and add a description.
  • Ports Tab: You can create input/output ports
  • Properties Tab: You can define the transaction control expression and tracing level.
  • Metadata Extensions Tab: You can add metadata information.
Transaction Control Expression
You can enter the transaction control expression in the Transaction Control Condition option in the properties tab. The transaction control expression uses the IIF function to test each row against the condition. Use the following syntax for the expression
Syntax: 
IIF (condition, value1, value2)

Example: 
IIF(dept_id=10, TC_COMMIT_BEFORE,TC_ROLLBACK_BEFORE)
Use the following built-in variables in the expression editor of the transaction control transformation:
  • TC_CONTINUE_TRANSACTION: The Integration Service does not perform any transaction change for this row. This is the default value of the expression.
  • TC_COMMIT_BEFORE: The Integration Service commits the transaction, begins a new transaction, and writes the current row to the target. The current row is in the new transaction.
  • TC_COMMIT_AFTER: The Integration Service writes the current row to the target, commits the transaction, and begins a new transaction. The current row is in the committed transaction.
  • TC_ROLLBACK_BEFORE: The Integration Service rolls back the current transaction, begins a new transaction, and writes the current row to the target. The current row is in the new transaction.
  • TC_ROLLBACK_AFTER: The Integration Service writes the current row to the target, rolls back the transaction, and begins a new transaction. The current row is in the rolled back transaction.
If the transaction control transformation evaluates to a value other than the commit, rollback or continue, then the integration service fails the session.
Transaction Control Transformation in Mapping
Transaction control transformation defines or redefines the transaction boundaries in a mapping. It creates a new transaction boundary or drops any incoming transaction boundary coming from upstream active source or transaction control transformation.
Transaction control transformation can be effective or ineffective for the downstream transformations and targets in the mapping. The transaction control transformation can become ineffective for downstream transformations or targets if you have used transformation that drops the incoming transaction boundaries after it. The following transformations drop the transaction boundaries.
  • Aggregator transformation with Transformation scope as "All Input".
  • Joiner transformation with Transformation scope as "All Input".
  • Rank transformation with Transformation scope as "All Input".
  • Sorter transformation with Transformation scope as "All Input".
  • Custom transformation with Transformation scope as "All Input".
  • Custom transformation configured to generate transactions
  • Transaction Control transformation
  • A multiple input group transformation, such as a Custom transformation, connected to multiple upstream transaction control points.
Mapping Guidelines and Validation
Use the following rules and guidelines when you create a mapping with a Transaction Control transformation:
  • If the mapping includes an XML target, and you choose to append or create a new document on commit, the input groups must receive data from the same transaction control point.
  • Transaction Control transformations connected to any target other than relational, XML, or dynamic MQSeries targets are ineffective for those targets.
  • You must connect each target instance to a Transaction Control transformation.
  • You can connect multiple targets to a single Transaction Control transformation.
  • You can connect only one effective Transaction Control transformation to a target.
  • You cannot place a Transaction Control transformation in a pipeline branch that starts with a Sequence Generator transformation.
  • If you use a dynamic Lookup transformation and a Transaction Control transformation in the same mapping, a rolled-back transaction might result in unsynchronized target data.
  • A Transaction Control transformation may be effective for one target and ineffective for another target. If each target is connected to an effective Transaction Control transformation, the mapping is valid.
  • Either all targets or none of the targets in the mapping should be connected to an effective Transaction Control transformation.
 

Informatica online training - Informatica Jobs

PMCMD Command Usage in Informatica

Informatica provides four built-in command line programs or utilities to interact with the informatica features. They are:
  • infacmd
  • infasetup
  • pmcmd
  • pmrep
This article covers only about the pmcmd command. The pmcmd is a command line utility provided by the informatica to perform the following tasks.
  • Start workflows.
  • Start workflow from a specific task.
  • Stop, Abort workflows and Sessions.
  • Schedule the workflows.
How to use PMCMD Command in Informatica:
1. Scheduling the workflow
The pmcmd command syntax for scheduling the workflow is shown below:
pmcmd scheduleworkflow -service informatica-integration-Service -d domain-name -u user-name -p password -f folder-name -w workflow-name
You cannot specify the scheduling options here. This command just schedules the workflow for the next run.
2. Start workflow
The following pmcmd command starts the specified workflow:
pmcmd startworkflow -service informatica-integration-Service -d domain-name -u user-name -p password -f folder-name -w workflow-name
3. Stop workflow
Pmcmd command to stop the infromatica workflow is shown below:
pmcmd stopworkflow -service informatica-integration-Service -d domain-name -u user-name -p password -f folder-name -w workflow-name
4. Start workflow from a task
You can start the workflow from a specified task. This is shown below:
pmcmd startask -service informatica-integration-Service -d domain-name -u user-name -p password -f folder-name -w workflow-name -startfrom task-name
5. Stopping a task.
The following pmcmd command stops the specified task instance:
pmcmd stoptask -service informatica-integration-Service -d domain-name -u user-name -p password -f folder-name -w workflow-name task-name
6. Aborting workflow and task.
The following pmcmd commands are used to abort workflow and task in a workflow:
pmcmd abortworkflow -service informatica-integration-Service -d domain-name -u user-name -p password -f folder-name -w workflow-name
Informatica online training - Informatica Jobs
pmcmd aborttask -service informatica-integration-Service -d domain-name

Informatica Problems With Solutions - Part 1

1. In this problem we will see how to implement the not equal operator, greater than, greater than or equal to, less than and less than or equal to operators when joining two tables in informatica.
Consider the below sales table as an example?
Table name: Sales
product, prod_quantity, price , Year
A         , 10                 , 100  , 2010
B         , 15                 , 150  , 2010
A         , 8                   , 80    , 2011
B         , 26                 , 260  , 2011
Now the problem is to identify the products whose sales is less than in the current year (In this example: 2011) when compared to the last year.
Here in this example, Product A sold less in 2011 when compared with the sales in 2010.
This problem can be easily implemented with the help of SQL query as shown below
SELECT  cy.*
FROM    SALES cy,
SALES py
WHERE   cy.product = py.product
AND        cy.year=2011
AND        py.year=2010
AND       cy.prod_quantity < py.prod_quantity;
In informatica, you can specify only equal to condition in joiner. Now we will see how to implement this problem using informatica.
Solution:
STEP1: Connect two source qualifier transformations to the source definition. Call the first source qualifier transformation as sq_cy  (cy means current year) and the other as sq_py  (py means previous year).
STEP2: In the sq_cy source qualifier transformation, specify the source filter as price=2011. In the sq_py, specify the source filter as price=2010
STEP3: Now connect these two source qualifier transformations to joiner transformation and make sq_cy as master, sq_py as detail. In the join condition, select the product port from master and detail.
STEP4: Now connect all the master ports and only the prod_quantity port from detail to the filter transformation. In the filter transformation specify the filter condition as prod_quantity < prod_quantity1. Here pord_quantity port is from master port and prod_quantity1 is from detail port.

Informatica online training - Informatica Jobs
STEP4: Connect all the ports except the prod_quantity1 of filter transformation to the target definition.
2. How to implement the not exists operator in informatica which is available in database?
Solution:
Implementing the Not Exists operator is very easy in informatica. For example, we want to get only the records which are available in table A and not in table B. For this use a joiner transformation with A as master and B as detail. Specify the join condition and in the join type, select detail outer join. This will get all the records from A table and only the matching records from B table.
Connect the joiner to a filter transformation and specify the filter condition as B_port is NULL. This will give the records which are in A and not in B. Then connect the filter to the target definition.

Informatica Real Time Scenarios - Solutions

This is continuation to my previous post on Informatica Real Time Complex scenarios which contains around 50 problems. Here i am providing few more real time Informatica scenarios with answers.
Informatica Real Time Scenarios:
Q1) Alternate Target Loading
My source is a flat file which contains N number of records. I want to load the source data into two targets such that first five records should loaded into the first target, next five records into the second target table. Again the next source five records into the first target table and so on. How to implement a Informatica mapping logic for this?
Solution:
  • Connect the source qualifier transformation to the expression transformation. In the expression transformation, create the below additional ports:
v_cnt (variable port) = v_cnt+1
o_cnt (output port) = v_cnt
  • Connect the expression transformation to the router transformation. Create two output groups in the router transformation and specify the following filter conditions:
--Filter condition for first output group
DECODE(substr(o_cnt,-1,1),1,TRUE,2,TRUE,3,TRUE,4,TRUE,5,TRUE,FALSE)
--Filter condition for second output group
DECODE(substr(o_cnt,-1,1),6,TRUE,7,TRUE,8,TRUE,9,TRUE,0,TRUE,FALSE)
  • Connect the router transformation output groups to the appropriate targets.
Q2) Load source data in multiple session run.
I have flat file as a source which contains N number of records. My requirement is to load half of the source data into the target table in the first session run and the remaining half of the records in the second session run. Create Informatica mapping to implement this logic? Assume that the source data does not change between session runs.
Solution:
  • Create a mapping to find out the number of records in the source and write the count to a parameter file. Let call this parameter as $$SOURCE_COUNT.
  • Create another mapping. Go to the mapping parameters and variables, create a mapping variable ($$VAR_SESSION_RUNS) with integer data type.
  • Connect the source qualifier transformation to the expression transformation. In the expression transformation, create the below additional ports.
v_Count (variable port) = v_Count+1
O_Run_flag (output port) = IIF($$vAR_SESSION_RUNS=0, 
                                  setvariable($$vAR_SESSION_RUNS,1), 
      IIF( !ISNULL($$vAR_SESSION_RUNS) 
                                          and v_Count=1,
                                              2, 
                                              $$vAR_SESSION_RUNS) 
          )
O_count (output port) = V_Count
  • Connect the expression transformation to the filter transformation and specify the following filter condition:
IIF (O_Run_Flag =1, v_count<= $$SOURCE_COUNT/2,
IIF (O_Run_Flag =2, v_count > $$SOURCE_COUNT/2))
  • Connect the filter transformation to the target.
  • Here i am assuming that you know how to use a parameter file. That is why I did not specify the complete details.
Informatica online training - Informatica Jobs

Dynamic Target Flat File Name Generation in Informatica

Informatica 9.x or later versions provides a feature for generating the target files dynamically. This feature allows you to
  • Create a new file for every session run
  • create a new file for each transaction.
Informatica provides a special port,"FileName" in the Target file definition. This port you have to add explicitly. See the below diagram for adding the "FileName" port.
Go to the Target Designer or Warehouse builder and edit the file definition. You have to click on the button indicated in red color circle to add the special port.
Now we will see some informatica mapping examples for creating the target file name dynamically and load the data.
1. Generate a new file for every session run.
Whenever the session runs you need to create a new file dynamically and load the source data into that file. To do this just follow the below steps:
STEP1: Connect the source qualifier to an expression transformation. In the expression transformation create an output port (call it as File_Name) and assign the expression as 'EMP_'||to_char(sessstarttime, 'YYYYMMDDHH24MISS')||'.dat'
STPE2: Now connect the expression transformation to the target and connect eh File_Name port of expression transformation to the FileName port of the target file definition.
STEP3: Create a workflow and run the workflow.
Here I have used sessstarttime, as it is constant throughout the session run. If you have used sysdate, a new file will be created whenever a new transaction occurs in the session run.
The target file names created would look like EMP_20120101125040.dat.
2. Create a new file for every session run. The file name should contain suffix as numbers (EMP_n.dat)
In the above mapping scenario, the target flat file name contains the suffix as 'timestamp.dat'. Here we have to create the suffix as a number. So, the file names should looks as EMP_1.dat, EMP_2.dat and so on. Follow the below steps:
STPE1: Go the mappings parameters and variables -> Create a new variable, $$COUNT_VAR and its data type should be Integer
STPE2: Connect the source Qualifier to the expression transformation. In the expression transformation create the following new ports and assign the expressions.
v_count (variable port) = v_count+1
v_file_count (variable port) = IIF(v_count = 1, SETVARIABLE($$COUNT_VAR,$$COUNT_VAR+1),$$COUNT_VAR)
o_file_name (output port) = 'EMP_'||v_file_count||'.dat'
STEP3: Now connect the expression transformation to the target and connect the o_file_name port of expression transformation to the FileName port of the target.
3. Create a new file once a day.
You can create a new file only once in a day and can run the session multiple times in the day to load the data. You can either overwrite the file or append the new data.
This is similar to the first problem. Just change the expression in expression transformation to 'EMP_'||to_char(sessstarttime, 'YYYYMMDD')||'.dat'. To avoid overwriting the file, use Append If Exists option in the session properties.
4. Create a flat file based on the values in a port.
You can create a new file for each distinct values in a port. As an example consider the employees table as the source. I want to create a file for each department id and load the appropriate data into the files.
STEP1: Sort the data on department_id. You can either use the source qualifier or sorter transformation to sort the data.
STEP2: Connect to the expression transformation. In the expression transformation create the below ports and assign expressions.
v_curr_dept_id (variable port) = dept_id
v_flag (variable port) = IIF(v_curr_dept_id=v_prev_dept_id,0,1)
v_prev_dept_id (variable port) = dept_id
o_flag (output port) = v_flag
o_file_name (output port) = dept_id||'.dat'
STEP4: Now connect the expression transformation to the transaction control transformation and specify the transaction control condition as
IIF(o_flag = 1, TC_COMMIT_BEFORE, TC_CONTINUE_TRANSACTION)
STEP5: Now connect to the target file definition.
Informatica online training - Informatica Jobs

Reverse the Contents of Flat File – Informatica

Q1) I have a flat file, want to reverse the contents of the flat file which means the first record should come as last record and last record should come as first record and load into the target file.As an example consider the source flat file data as
Informatica Enterprise Solution
Informatica Power center
Informatica Power exchange
Informatica Data quality
The target flat file data should look as
Informatica Data quality
Informatica Power exchange
Informatica Power center
Informatica Enterprise Solution
Solution:
Follow the below steps for creating the mapping logic
  • Create a new mapping.
  • Drag the flat file source into the mapping.
  • Create an expression transformation and drag the ports of source qualifier transformation into the expression transformation.
  • Create the below additional ports in the expression transformation and assign the corresponding expressions
Variable port: v_count = v_count+1
Output port o_count = v_count
  • Now create a sorter transformation and drag the ports of expression transformation into it.
  • In the sorter transformation specify the sort key as o_count and sort order as DESCENDING.
  • Drag the target definition into the mapping and connect the ports of sorter transformation to the target.
Q2) Load the header record of the flat file into first target, footer record into second target and the remaining records into the third target.
The solution to this problem I have already posted by using aggregator and joiner. Now we will see how to implement this by reversing the contents of the file.
Solution:
  • Connect the source qualifier transformation to the expression transformation. In the expression transformation create the additional ports as mentioned above.
  • Connect the expression transformation to a router. In the router transformation create an output group and specify the group condition as o_count=1. Connect this output group to a target and the default group to sorter transformation.
  • Sort the data in descending order on o_count port.
  • Connect the output of sorter transformation to expression transformation (don’t connect o_count port).
  • Again in the expression transformation create the same additional ports mentioned above.
  • Connect this expression transformation to router and create an output group. In the output group specify the condition as o_count=1 and connect this group to second target. Connect the default group to the third group.
Informatica online training - Informatica Jobs

Generate rows based on a column value - Informatica

Q) How to generate or load values in to the target table based on a column value using informatica etl tool.
I have the products table as the source and the data of the products table is shown below.
Table Name: Products
Product  Quantity
-----------------
Samsung  NULL
Iphone   3
LG       0
Nokia    4
Now i want to duplicate or repeat each product in the source table as many times as the value in the quantity column. The output is
product  Quantity
----------------
Iphone   3
Iphone   3
Iphone   3
Nokia    4
Nokia    4
Nokia    4
Nokia    4
The Samsung and LG products should not be loaded as their quantity is NULL, 0 respectively.
Now create informatica workflow to load the data in to the target table?
Solution:
Follow the below steps
  • Create a new mapping in the mapping designer
  • Drag the source definition in to the mapping
  • Create the java transformation in active mode
  • Drag the ports of source qualifier transformation in to the java transformation.
  • Now edit the java transformation by double clicking on the title bar of the java transformation and go to the "Java Code" tab.
  • Enter the below java code in the "Java Code" tab.

if (!isNull("quantity"))
{
  double cnt = quantity;
  for (int i = 1; i <= quantity; i++)
  {
    product = product;
    quantity = quantity;
    generateRow();
  }
}

informatica joiner transformation example

  • Now compile the java code. The compile button is shown in red circle in the image.
  • Connect the ports of the java transformation to the target.
  • Save the mapping, create a workflow and run the workflow.
Informatica online training - Informatica Jobs

Target update override - Informatica

When you used an update strategy transformation in the mapping or specified the "Treat Source Rows As" option as update, informatica integration service updates the row in the target table whenever there is match of primary key in the target table found.
The update strategy works only
  • when there is primary key defined in the target definition.
  • When you want update the target table based on the primary key.
What if you want to update the target table by a matching column other than the primary key? In this case the update strategy wont work. Informatica provides feature, "Target Update Override", to update even on the columns that are not primary key.
You can find the Target Update Override option in the target definition properties tab. The syntax of update statement to be specified in Target Update Override is
UDATE TARGET_TABLE_NAME
SET TARGET_COLUMN1 = :TU.TARGET_PORT1,
    [Additional update columns]
WHERE TARGET_COLUMN = :TU.TARGET_PORT
AND   [Additional conditions]
Here TU means target update and used to specify the target ports.
Example: Consider the employees table as an example. In the employees table, the primary key is employee_id. Let say we want to update the salary of the employees whose employee name is MARK. In this case we have to use the target update override. The update statement to be specified is
UPDATE EMPLOYEES
SET SALARY = :TU.SAL
WHERE EMPLOYEE_NAME = :TU.EMP_NAME

Informatica online training - Informatica Jobs

SQL Query Overwrite in Source Qualifier - Informatica

One of the properties of source qualifier transformation is "SQL Query" which can be used to overwrite the default query with our customized query. We can generate SQL queries only for relational sources. For flat files, all the properties of source qualifier transformation will be disabled state.
Here we will see how to generate the SQL query and the errors that we will get while generating the SQL query.
Error When Generating SQL query:
The most frequent error that we will get is "Cannot generate query because there are no valid fields projected from the Source Qualifier".
First we will see simulate this error and then we will see how to avoid this. Follow the below steps for simulating and fixing error:
  • Create a new mapping and drag the relational source into it. For example drag the customers source definition into the mapping.
  • Do not connect the source qualifier transformation to any of other transformations or target.
  • Edit the source qualifier and go to the properties tab and then open the SQL Query Editor.
  • Enter the ODBC data source name, user name, password and then click on Generate SQL.
  • Now we will get the error while generating the SQL query.
  • Informatica produces this error because the source qualifier transformation ports are not connected to any other transformations or target. Informatica just knows the structure of the source. However it doesn't know what columns to be read from source table. It will know only when the source qualifier is connected to downstream transformations or target.
  • To avoid this error, connect the source qualifier transformation to downstream transformation or target.
Generating the SQL Query in Source Qualifier:
To explain this I am taking the customers table as the source. The source structure looks as below
Create table Customers
(
Customer_Id Number,
Name        Varchar2(30),
Email_Id    Varchar2(30),
Phone       Number
)
Follow the below steps to generate the SQL query in source qualifier transformation.
  • Create a new mapping and drag the customers relational source into the mapping.
  • Now connect the source qualifier transformation to any other transformation or target. Here I have connected the SQ to expression transformation. This is shown in the below image.
  • Edit the source qualifier transformation, go to the properties tab and then open the editor of SQL query.
  • Enter the username, password, data source name and click on Generate SQL query. Now the SQL query will be generated. This is shown in the below image.
The SQL query generated is
SELECT Customers.Customer_Id,
        Customers.Name,
 Customers.Email_Id,
 Customers.Phone
FROM Customers
Now we will do a small change to understand more about the "Generating SQL query". Remove the link (connection) between Name port of source qualifier and expression transformation.
Repeat the above steps to generate the SQL query and observe what SQL query will be generated.
The SQL query generated in this case is
SELECT Customers.Customer_Id,
 Customers.Email_Id,
 Customers.Phone
FROM Customers
The Name column is missing in the generated query. This means that whatever the ports connected from Source Qualifier transformation to other downstream transformations or target will be included in the SQL query and read from the database table.

Avoiding Sequence Generator Transformation in Informatica

Q) How to generate sequence numbers without using the sequence generator transformation?
We use sequence generator transformation mostly in SCDs. Using a sequence generator transformation to generate unique primary key values can cause performance issues as an additional transformation is required to process in mapping.
You can use expression transformation to generate surrogate keys in a dimensional table. Here we will see the logic on how to generate sequence numbers with expression transformation.
Sequence Generator Reset Option:
When you use the reset option in a sequence generator transformation, the sequence generator uses the original value of Current Value to generate the numbers. The sequences will always start from the same number.
As an example, if the Current Value is 1 with reset option checked, then the sequences will always start from value 1 for multiple session runs. We will see how to implement this reset option with expression transformation.
Follow the below steps:
  • Create a mapping parameter and call it as $$Current_Value. Assign the default value to this parameter, which is the start value of the sequence numbers.
  • Now create an expression transformation and connect the source qualifier transformation ports to the expression transformation.
  • In the expression transformation create the below additional ports and assign the expressions:
v_seq (variable port) = IIF(v_seq>0,v_seq+1,$$Current_Value)
o_key (output port) = v_seq
  • The v_seq port generates the numbers same as NEXTVAL port in sequence generator transformation.
Primary Key Values Using Expression and Parameter:
We will see here how to generate the primary key values using the expression transformation and a parameter. Follow the below steps:
  • Create a mapping to write the maximum value of primary key in the target to a parameter file. Assign the maximum value to the parameter ($$MAX_VAL) in this mapping. Create a session for this mapping. This should be the first session in the workflow.
  • Create another mapping where you want to generate the sequence numbers. In this mapping, connect the required ports to the expression transformation, create the below additional ports in the expression transformation and assign the below expressions:
v_cnt (variable port) = v_cnt+1
v_seq (variable port) = IIF( ISNULL($$MAX_VAL) OR $$MAX_VAL=0,1,v_cnt+$$MAX_VAL)
o_surrogate_key (output port) = v_seq
  • The o_surrogate_key port generates the primary key values just as the sequence generator transformation.
Primary Key Values Using Expression and Lookup Transformations:
Follow the below steps to generate sequence numbers using expression and lookup transformations.
  • Create an unconnected lookup transformation with lookup table as target. Create a primary_key_column port with type as output/lookup/return in the lookup ports tab. Create another port input_id with type as input. Now overwrite the lookup query to get the maximum value of primary key from the target. The query looks as
SELECT MAX(primary_key_column) FROM Dimension_table
  • Specify the lookup condition as primary_key_column >= input_id
  • Now create an expression transformation and connect the required ports to it. Now we will call the unconnected lookup transformation from this expression transformation. Create the below additional port in the expression transformation:
v_cnt (variable port) = v_cnt+1
v_max_val (variable port) = IIF(v_cnt=1, :LKP.lkp_trans(1), IIF(ISNULL(v_max_val) or v_max_val=0, 1, v_max_val))
v_seq (variable port) = IIF(ISNULL(v_max_val) or v_max_val=0, 1, v_cnt+v_max_val)
o_primary_key (output port) = v_seq
  • The o_primary_key port generates the surrogate key values for the dimension table.

Normalizer Transformation Error - Informatica

Normalizer transformation is used to convert the data in multiple columns into different rows. Basically the normalizer transformation converts the denormalized data in a table in to a normalized table.
Normalizer Transformation Error
Getting the following Error for the Normalizer transformation in mapping when pivoting the columns in to Rows
TT_11054 Normalizer Transformation: Initialization Error: [Cannot match OFOSid with IFOTid.]
How to fix the Normalizer Transformation Error?
Solution:
Follow the below steps to avoid this error.
  1. There should be no unconnected input ports to the Normalizer transformation.
  2. If the Normalizer has an OCCURS in it, make sure number of input ports matches the number of OCCURS.
Informatica online training - Informatica Jobs

Creating a Non Reusable Object from Reusable Object

Q) How to create a non-reusable transformation or session or task from a reusable transformation or session or task?
I still remember my first project in which i created so many reusable transformations and developed a mapping. My project lead reviewed the code and told me that you created unnecessary reusable transformation change them to non reusable transformations. I created non reusable transformations and re-implemented the entire logic. It took almost one day for me to implement the code. Still so many new informatica developers will do the same mistake and re implement the entire logic.
I found an easy way to create a non-reusable transformation from a reusable transformation. Follow the below steps to create a non-reusable transformation or session or task from a reusable transformation or session or task in informatica is
  1. Go to the Navigator which is on the left side.
  2. Select the reusable transformation or session or task which you want to convert to non resuable with the mouse.
  3. Drag the object (transformation/session/task) to the work-space and just before leaving the object on the work-space hold the ctrl key and then release the object.
Now you are done with creating a non-reusable transformation or session or task.

POWERCENTER CLIENT

The Power Center Client consists of the following applications that we use to manage the repository, design mappings, mapplets, and create sessions to load the data:
  1. Designer
  2. Data Stencil
  3. Repository Manager
  4. Workflow Manager
  5. Workflow Monitor
1. Designer:
Use the Designer to create mappings that contain transformation instructions for the Integration Service.
The Designer has the following tools that you use to analyze sources, design target Schemas, and build source-to-target mappings:
  •  Source Analyzer: Import or create source definitions.
  •  Target Designer: Import or create target definitions.
  •  Transformation Developer: Develop transformations to use in mappings.
You can also develop user-defined functions to use in expressions.
  •  Mapplet Designer: Create sets of transformations to use in mappings.
  •  Mapping Designer: Create mappings that the Integration Service uses to Extract, transform, and load data.
clip_image002
2.Data Stencil
Use the Data Stencil to create mapping template that can be used to generate multiple mappings. Data Stencil uses the Microsoft Office Visio interface to create mapping templates. Not used by a developer usually.
3.Repository Manager
Use the Repository Manager to administer repositories. You can navigate through multiple folders and repositories, and complete the following tasks:
Informatica online training - Informatica Jobs
  • Manage users and groups: Create, edit, and delete repository users and User groups. We can assign and revoke repository privileges and folder Permissions.
  • Perform folder functions: Create, edit, copy, and delete folders. Work we perform in the Designer and Workflow Manager is stored in folders. If we want to share metadata, you can configure a folder to be shared.
  • View metadata: Analyze sources, targets, mappings, and shortcut dependencies, search by keyword, and view the properties of repository Objects. We create repository objects using the Designer and Workflow Manager Client tools.
We can view the following objects in the Navigator window of the Repository Manager:
  • Source definitions: Definitions of database objects (tables, views, synonyms) or Files that provide source data.
  • Target definitions: Definitions of database objects or files that contain the target data.
  • Mappings: A set of source and target definitions along with transformations containing business logic that you build into the transformation. These are the instructions that the Integration Service uses to transform and move data.
  • Reusable transformations: Transformations that we use in multiple mappings.
  • Mapplets: A set of transformations that you use in multiple mappings.
  • Sessions and workflows: Sessions and workflows store information about how and When the Integration Service moves data. A workflow is a set of instructions that Describes how and when to run tasks related to extracting, transforming, and loading Data. A session is a type of task that you can put in a workflow. Each session Corresponds to a single mapping.
4.Workflow Manager :
Use the Workflow Manager to create, schedule, and run workflows. A workflow is a set of instructions that describes how and when to run tasks related to extracting, transforming, and loading data.
The Workflow Manager has the following tools to help us develop a workflow:
  • Task Developer: Create tasks we want to accomplish in the workflow.
  • Work let Designer: Create a worklet in the Worklet Designer. A worklet is an object that groups a set of tasks. A worklet is similar to a workflow, but without scheduling information. We can nest worklets inside a workflow.
  • Workflow Designer: Create a workflow by connecting tasks with links in the Workflow Designer. You can also create tasks in the Workflow Designer as you develop the workflow.
Informatica online training - Informatica Jobs
When we create a workflow in the Workflow Designer, we add tasks to the workflow. The Workflow Manager includes tasks, such as the Session task, the Command task, and the Email task so you can design a workflow. The Session task is based on a mapping we build in the Designer.
We then connect tasks with links to specify the order of execution for the tasks we created. Use conditional links and workflow variables to create branches in the workflow.
5.Workflow Monitor
Use the Workflow Monitor to monitor scheduled and running workflows for each Integration Service. We can view details about a workflow or task in Gantt chart view or Task view. We Can run, stop, abort, and resume workflows from the Workflow Monitor. We can view Sessions and workflow log events in the Workflow Monitor Log Viewer.
The Workflow Monitor displays workflows that have run at least once. The Workflow Monitor continuously receives information from the Integration Service and Repository Service. It also fetches information from the repository to display historic Information.
Informatica online training - Informatica Jobs
 

Services Behind Scene

INTEGRATION SERVICE PROCESS
The Integration Service starts an Integration Service process to run and monitor workflows. The Integration Service process accepts requests from the Power Center Client and from pmcmd. It performs the following tasks:
  • Manages workflow scheduling.
  • Locks and reads the workflow.
  • Reads the parameter file.
  • Creates the workflow log.
  • Runs workflow tasks and evaluates the conditional links connecting tasks.
  • Starts the DTM process or processes to run the session.
  • Writes historical run information to the repository.
  • Sends post-session email in the event of a DTM failure.
LOAD BALANCER
The Load Balancer is a component of the Integration Service that dispatches tasks to achieve optimal performance and scalability. When we run a workflow, the Load Balancer dispatches the Session, Command, and predefined Event-Wait tasks within the workflow.
The Load Balancer dispatches tasks in the order it receives them. When the Load Balancer needs to dispatch more Session and Command tasks than the Integration Service can run, it places the tasks it cannot run in a queue. When nodes become available, the Load Balancer dispatches tasks from the queue in the order determined by the workflow service level.
DTM PROCESS
Informatica online training - Informatica Jobs
When the workflow reaches a session, the Integration Service process starts the DTM process. The DTM is the process associated with the session task. The DTM process performs the following tasks:
  • Retrieves and validates session information from the repository.
  • Performs pushdown optimization when the session is configured for pushdown optimization.
  • Adds partitions to the session when the session is configured for dynamic partitioning.
  • Expands the service process variables, session parameters, and mapping variables and parameters.
  • Creates the session log.
  • Validates source and target code pages.
  • Verifies connection object permissions.
  • Runs pre-session shell commands, stored procedures, and SQL.
  • Sends a request to start worker DTM processes on other nodes when the session is configured to run on a grid.
  • Creates and run mapping, reader, writer, and transformation threads to extract, transform, and load data.
  • Runs post-session stored procedures, SQL, and shell commands.
  • Sends post-session email.
PROCESSING THREADS
The DTM allocates process memory for the session and divides it into buffers. This is also known as buffer memory. The default memory allocation is 12,000,000 bytes.
The DTM uses multiple threads to process data in a session. The main DTM thread is called the master thread.
The master thread can create the following types of threads:
CODE PAGES and DATA MOVEMENT
A code page contains the encoding to specify characters in a set of one or more languages. An encoding is the assignment of a number to a character in the character set.
The Integration Service can move data in either ASCII or Unicode data movement mode. These modes determine how the Integration Service handles character data.
We choose the data movement mode in the Integration Service configuration settings. If we want to move multi byte data, choose Unicode data movement mode.
ASCII Data Movement Mode: In ASCII mode, the Integration Service recognizes 7-bit ASCII and EBCDIC characters and stores each character in a single byte.
Unicode Data Movement Mode: Use Unicode data movement mode when sources or targets use 8-bit or multi byte character sets and contain character data.

Try your Hand’s on Admin-Console

Try your Hand’s on Admin-Console

Repository Manager Tasks:
Add a repository to the Navigator, and then configure the domain connection information when we connect to the repository.
1.Adding a Repository to the Navigator :
1. In any of the Power Center Client tools, click Repository > Add.
clip_image002

2. Enter the name of the repository and a valid repository user name.
3. Click OK.
Before we can connect to the repository for the first time, we must configure the Connection information for the domain that the repository belongs to.
clip_image004clip_image007clip_image008clip_image030clip_image031

2.Configuring a Domain Connection
  1. In a Power Center Client tool, select the Repositories node in the Navigator.
  2. Click Repository > Configure Domains to open the Configure Domains dialog box.
  3. Click the Add button. The Add Domain dialog box appears.
  4. Enter the domain name, gateway host name, and gateway port number.
  5. Click OK to add the domain connection.
Informatica online training - Informatica Jobs
3.Connecting to a Repository
  1. Launch a Power Center Client tool.
  2. Select the repository in the Navigator and click Repository > Connect, or double-click the repository.
  3. Enter a valid repository user name and password.
  4. Click Connect.
Click on more button to add, change or view domain information.
clip_image033clip_image014[1]
4.Viewing Object Dependencies
Before we change or delete repository objects, we can view dependencies to see the impact on other objects. For example, before you remove a session, we can find out which workflows use the session. We can view dependencies for repository objects in the Repository Manager, Workflow Manager, and Designer tools.
Steps:
  1. Connect to the repository.
  2. Select the object of use in navigator.
  3. Click Analyze and Select the dependency we want to view.
5.Validating Multiple Objects
We can validate multiple objects in the repository without fetching them into the workspace. We can save and optionally check in objects that change from invalid to valid status as a result of the validation. We can validate sessions, mappings, mapplets, workflows, and worklets.
Steps:
  1. Select the objects you want to validate.
  2. Click Analyze and Select Validate
  3. Select validation options from the Validate Objects dialog box
  4. Click Validate.
  5. Click a link to view the objects in the results group.
6.Comparing Repository Objects
We can compare two repository objects of the same type to identify differences between the objects. For example, we can compare two sessions to check for differences. When we compare two objects, the Repository Manager displays their attributes.
Steps:
  1. In the Repository Manager, connect to the repository.
  2. In the Navigator, select the object you want to compare.
  3. Click Edit > Compare Objects.
  4. Click Compare in the dialog box displayed.
7.Truncating Workflow and Session Log Entries
When we configure a session or workflow to archive session logs or workflow logs, the Integration Service saves those logs in local directories. The repository also creates an entry for each saved workflow log and session log. If we move or delete a session log or workflow log from the workflow log directory or session log directory, we can remove the entries from the repository.
Steps:
  1. In the Repository Manager, select the workflow in the Navigator window or in the Main window.
  2. Choose Edit > Truncate Log. The Truncate Workflow Log dialog box appears.
  3. Choose to delete all workflow and session log entries or to delete all workflow and session log entries with an end time before a particular date.
  4. If you want to delete all entries older than a certain date, enter the date and time.
  5. Click OK.
8.Managing User Connections and Locks
In the Repository Manager, we can view and manage the following items:
Repository object locks: The repository locks repository objects and folders by user. The repository creates different types of locks depending on the task. The Repository Service locks and unlocks all objects in the repository.
User connections: Use the Repository Manager to monitor user connections to the repository. We can end connections when necessary.
Types of locks created:
1. In-use lock: Placed on objects we want to view
2. Write-intent lock: Placed on objects we want to modify.
3. Execute lock: Locks objects we want to run, such as workflows and sessions
Steps:
  1. Launch the Repository Manager and connect to the repository.
  2. Click Edit > Show User Connections or Show locks
  3. The locks or user connections will be displayed in a window.
  4. We can do the rest as per our need.
9.Managing Users and Groups
  1. In the Repository Manager, connect to a repository.
  2. Click Security > Manage Users and Privileges.
  3. Click the Groups tab to create Groups. or
  4. Click the Users tab to create Users
  5. Click the Privileges tab to give permissions to groups and users.
  6. Select the options available to add, edit, and remove users and groups.
There are two default repository user groups:
Administrators: This group initially contains two users that are created by default.
The default users are Administrator and the database user that created the repository. We cannot delete these users from the repository or remove them from the Administrators group.
Public: The Repository Manager does not create any default users in the Public group.
10 Working with Folders
We can create, Edit or delete folder as per our need.
  1. In the Repository Manager, connect to a repository.
  2. Click Folder > Create.
Enter the following information:
clip_image035

3. Click ok.
Informatica online training - Informatica Jobs