Oracle Data Integrator

Oracle Data Integrator Point of View

Created by Name Role Signature Date

Reviewed by

Approved By

Nishant Jain Sr. Architect Nishant July 23, 2013

C3-Protected

Page 1

Oracle Data Integrator Preface

Data Integration plays a vital role in this technology driven market. Every organization maintains several operational systems, whose data requires consolidation for decision-making. The concept is common from last many decades but data integration paradigm has changed a lot with multifold data volume. ETL attracted lot of attention when organizations were struggling to integrate data across heterogeneous systems. Over the period of time, ETL is now the obvious choice for any data integration activity. There are many ETL/ELT tools available in the market offered by different vendors and there is no straight forward rule on which one is better but there are certainly few parameters based on which fit-for-purpose tool can be selected ETL/ELT tools are quite expensive but choosing a wrong tool for an application will be more expensive. In this document, Oracle BI CoE will share their point of view on Oracle Data Integrator tool (ODI), which is an E-LT tool.

C3-Protected

Page 2

Oracle Data Integrator Contents

Preface .......................................................................................................................................................... 2 Introduction .................................................................................................................................................. 4 Oracle Data Integrator Capabilities............................................................................................................... 5 Product Roadmap ......................................................................................................................................... 5 Does ODI fit in your situation?...................................................................................................................... 6 Limitations .................................................................................................................................................... 7 Summary ....................................................................................................................................................... 7

C3-Protected

Page 3

Oracle Data Integrator Introduction

Oracle offers ODI (Oracle Data Integrator) as a data integration tool in Fusion Middleware stack based on high performance E-LT(Extract Load and Transform) architecture. Being an E-LT it is a database agnostic tool (like its predecessor OWB) and instead of moving all the data through an intermediate ETL transformation server, the E-LT approach leverages target database to perform the transformations, improves performance at much lower total cost of ownership. Data goes straight from sources to targets and transformed many times faster than other tools. Unlike the Pushdown Processing features recently introduced by the conventional ETL providers, Oracle's native E-LT architecture is not a mere after-thought. Most of these so-called "pushdown optimized" transformations still occur inside the ETL engines and requires the physical data to transit over the network and through their engines anyway. While pushdown optimization may indeed improve performance of their classical ETL processes in some cases, the Oracle E-LT architecture is in a performance class of its own.

C3-Protected

Page 4

Oracle Data Integrator Oracle Data Integrator Capabilities

Before evaluating ODI on its strength and weakness, let see what makes this tool unique. Out-of-box Knowledge Modules: Oracle provides KMs, which are extendible as per application requirement. ODI also accept custom-made KM and their reusability reduce the build effort to a great extent. KM contains ready-to-use code, based on a number of data integration scenarios. Therefore, no more brainstorming and effort to develop the logic unless there is any specific application requirement Change Data Capture: ODI comes along with off-the-shelf CDC framework for Oracle and nonOracle database. In few clicks, an application will be CDC ready without any extra effort SDK: Oracle offered ODI SDK, using which we can automate build phase itself. It can develop code on the fly based on metadata, which is almost impossible in any ETL tool Data Lineage: Oracle offers package for OBIEE-ODI data lineage to understand end-to-end data flow of the application. Similar information can be extracted from ODI repositories using simple SQL Data Quality: ODI offers inbuilt error workflow to identify and validate static table constraints with error recycling option. ODI also have inbuilt plugin for Oracle Enterprise Data Quality tool (OEDQ) to invoke OEDQ services High Availability: ODI Agent load balancing concept on Weblogic clusters can get the High availability and ensure Zero Downtime for the application Data Federation / Data Services: ODI have capability to generate and deploy Data and Web services on Weblogic. Few clicks can provide virtual, canonical views of data distributed over several systems through federated queries over heterogeneous sources.

Product Roadmap
In last couple of year, Oracle has added lot many functionalities in ODI, with a vision to establish this tool as a strategic data integration tool in its product family. Few major enhancements can clearly depict Oracles focus on this tool By replacing Informatica PowerCenter with ODI in OBIA 11.1.1.7 release, Oracle declared ODI as its strategic data integration tool for all upcoming Oracle products Recent ODI 11.1.1.7 release have extended compatibility with IBM Websphere, which can help ODI to promulgate in non-oracle applications ODI Load Plan replaced Data Warehouse Administrator Console (DAC), widely for scheduling processes in OBIA product range OWB to be diluted and merge with ODI, a strategic decision to have a unified data integration roadmap using ODI ODI with Exadata is the power pack solution by Oracle for all VLDB and Bigdata applications ODI seamless integration with Oracle GoldenGate (real-time data integration) and Oracle Enterprise Data Quality (Data Governance and Quality) Page 5

C3-Protected

Oracle Data Integrator Does ODI fit in your situation?

In addition to the capabilities mentioned earlier. Here are some more scenarios which can help to evaluation best fir-for-purpose ETL tool Data integration from homogeneous/heterogeneous source(s) to a single target o Development is very less in this situation as ODI ready-to-use KM can be used for any kind of operations e.g. Incremental Update, Insert-only, SCD . Development effort will be very less o If there are multiple targets then development effort would be more as more Interface need to be created and every interface will read the same data from source. An ODI interface can write to a single homogeneous target at a time Application demands control on the executable code o In a situation where client want to have a full control over the code, processing the data then ODI is the tool of choice as it provides transparency to all the code it submit for execution. o ODI can also generate the executable code for review without executing it and also make debugging an easy task o ODI also give turnaround time during troubleshooting by allowing code fix from logs Application hosting multiple technologies o In situation, where a very complex procedures are written in Java script/Bean or JMX then the same code can be re-used in ODI without any change (provided that technology supported by ODI) o Otherwise, it would be a massive task to migrate the code into ETL o Since ODI is an E-LT and leverage underlying technology for data processing. Therefore, it is not bounded for any specific language to write custom code. Coding can be done in any supporting language Real-Time Data Integration o ODI offers CDC framework for Oracle and non-oracle databases. Therefore, it is very quick to enable CDC in your application using ODI o Otherwise, it would be a mini-project in itself to write such framework from scratch

C3-Protected

Page 6

Oracle Data Integrator Limitations

Commit all or nothing o Unlike other ETL, which process row-by-row, ODI process data as per SQL statement and it will not commit until whole data written to database o If a single SQL trying to commit massive data then it will require large memory cache and we need to careful for such job if we are running other jobs in parallel o At Cognizant Oracle BI CoE, a KM has been developed for Oracle 11g onwards mitigating this issue Data cache feature is not available o ODI doesnt have its own engine and doesnt persit data anywhere, which can be reused by other processes o Every time we run an ODI process to lookup same data, it will hit database everytime

Summary
ODI is a great tool with clear strategic roadmap.Being an E-LT, there are few limitations but if any application which are not impacted by those then ODI is no-doubt a good choice of ETL. There is hardly any tool which allow to generate ETL code on the fly, flexible enough to customise the in-built code as per application requirement and so transparent that every single executable code is visible in the logs. All this is possible without procuring an extra server/CPU/middle-tier.

C3-Protected

Page 7