Professional Documents
Culture Documents
EAI vs. ETL: Drawing Boundaries For Data Integration: Applications
EAI vs. ETL: Drawing Boundaries For Data Integration: Applications
EAI vs. ETL: Drawing Boundaries For Data Integration: Applications
A W h i t e P A P e r S e r i e S
eAi And etL technoLogy hAve StrengthS
And WeAkneSSeS ALike. there Are
cLeAr boundArieS Around the tyPeS
of APPLicAtion integrAtion ProjectS
moSt APProPriAte for eAch technoLogy.
EAI vs. ETL: Drawing Boundaries for Data Integration
Business enterprises invest millions of dollars to implement
and deliver Data Warehousing and Business Intelligence (BI) initiatives
that rely on consistent, accurate and reliable data. IT organizations in these enterprises
must ensure that proper integration techniques are selected to address the data needs of the organization.
Positioning a common enterprise-wide integration strategy with EAI is essential to establish a clear-
cut partnership between business needs and IT solutions. Data integration, a function of ETL, is a
prominent need as mediocre data at the foundation of any BI initiative fails to provide an accurate picture
of the business. Thus the vital question: EAI or ETL?
In this paper, we'll explore this question, comparing ETL against the data integration element of EAI.
introduction
cALL for eAi
cALL for etL
WhAt iS eAi?
WhAt iS etL?
comPAring eAi And etL
diStinctive fActorS
eAi vS. etL: A deciSion mAking guide
drAWing boundArieS: eAi vS. etL
the bottom Line
2 0 0 7 S y nt e l , i nc.
1
2
3
4
5
6
7
8
9
eAi vs. etl: Drawing Boundaries for Data integration
1.
cAll for eAi
Most business activities involve multiple applica-
tions and information sources; incompatibilities
between these systems can cause delays and errors
that prevent organization from achieving real-time
business. The key to increasing operational effi-
ciency and maximizing the individual value of these
systems is ensuring that they can communicate and
interact in real time.
Some of the challenges facing modern
organizations are:
Giving the business complete, transparent
access to information
Enabling seamless movement of information
from one application to another
EAI, as a discipline, aims to alleviate many of these
problems as well as create new paradigms for truly
lean proactive organizations.
2.
cAll for etl
ETL (Extract, Transform and Load) is the tech-
nology with the focus for data integration, whether
in batch or real time for data stores/data ware-
houses. It synchronizes data between diverse appli-
cations and involves a lot more data manipulation
than simply moving data from point A to B. There
is reconciliation, cross matching, de-duping, cleans-
ing - all data-intensive tasks that lay the foundation
for facilitating analysis and reporting.
These systems are no longer stand-alone and sepa-
rate from operational processingthey are inte-
grated with overall business processes.
ETL is no longer nice to have, but is essential to
success.
EAI, as a discipline, aims to create
lean, proactive organizations.
EAI LEvELS
DATA-LEvEL EAI
the data-level eAi technique
implements information
exchange among multiple
application data stores using
traditional extract, transform,
and load (etL) techniques
that are commonplace in
data warehouse deploy-
ments.
MESSAgE-LEvEL EAI
message-level eAi manages
message exchange among
multiple applications using
reliable queuing systems.
PrOCESS-LEvEL EAI
Process-level eAi technique
goes beyond message-level
eAi by overlaying a work-
flow management capability
on top of message delivery
capability.
Integration Technologies Working in Concert
2EPORTING
&RAMEWORK
2EPORT0ORTAL
3ALES )NVENTORY
%4,
$7/$3
2EPORT
$ASHBOARD
Figure 1. Example of integration
technologies working together.
3.
whAt iS eAi?
Enteprise Application Integration is the process of
aligning a businesss strategic vision with its infor-
mation technology
Enterprise Application Integration (EAI) solu-
tions enable the automation of end-to-end business
processes by coordinating sequences of tasks and
resources (both systems and people) that perform
them. EAI solutions support sophisticated excep-
tion management and the dynamic modification of
processes even when processes are underway.
EAI involves developing a unified view of an
enterprises business and its applications, seeing
how existing applications fit into the new view, and
then devising ways to efficiently reuse what already
exists while adding new applications and data.
EAI provides packaged integration solutions
to help the enterprise develop a consistent
approach to integration for all applications.
4.
whAt iS etl?
Extract, Transform and Load (ETL) provides data consolidation
for building permanent databases used for analytics or reports,
data federation for creating virtual dashboards or reports, and data
propagation for the transfer of data between applications.
These three database functions are combined into one tool to pull
data out of source databases and place it into target databases.
ETL is used to migrate data from one or more databases to others,
to form data repositories, data marts, data warehouses and also to
convert databases from one format or type to another.
Extract - the process of reading data from source systems.
Data can be extracted in schedule-driven pull mode or event
driven push mode. Pull mode operation supports data consol-
idation and is typically done in batch. Push mode operation is
one online by propagating data changes to target data stores.
Transform - the process of converting the extracted data from
its existing form into the format it needs to be in so that it
can be placed into other systems or databases. Transformation
occurs by using rules or lookup tables or by combining the
data with other data.
Load - the process of creation and execution of workflows to
write data into the target systems. Data loading may cause
a complete refresh of a target data store or may be done by
updating the target destination. Interfaces here include de
facto standards like ODBC, JBDC, JMS, or application inter-
faces. Loads could be parallel, synchronized or sequenced;
e.g., ETL tool support parallel execution which dramatically
reduces response time for data-intensive operations on data
warehouses/data stores.
"USINESSTO"USINESS&LOW
Secure, lnternet-based llow ol data between businesses
0ROCESS&LOW
Process/workllow modeling, alarms and alerts, decision matrices.
)NFORMATION&LOW
Event-based data translormation
Pre-built application adapters (SAP, Oracle Apps, PeopleSolt)
$ATA&LOW
Peal-time change data capture, translorm and llow, publish and subscribe, meta data management
4RANSPORT&LOW
TCP/lP, SNA, MO Series Messaging
Figure 2. The EAI architecture has various layers that reflect an increasing level of matu-
rity in the integration environment with the overall enterprise application framework.
EAI is the process of aligning a business's strategic vision with its
information technology.
Admlnlstratlon
& Operatlons
Servlces
Transport
Servlces
Pun Tlme
Meta Data
Servlces
Extract
Load
Transform
Source Adapters
Target Adapters
Database
Fl|es
Legacy
App|lcatlons
Database
& Fl|es
Meta Data
Peposltory
Meta data
lmport/export
Figure 3. The ETL process
The other services which form an integral part of the ETL
framework are:
Administration and Operation services - these services
ensure effective utilization of resources in the data synchro-
nization environment. They ensure effective administration
through job scheduling and tracking, metadata management,
error recovery, etc.
Transport services - the process of moving raw or trans-
formed data from a source to a target system.
Metadata services - Metadata is descriptive information
about data and other structures, such as objects, business
rules, and processes that manipulate data. Metadata can be
grouped into two categories:
8.
DrAwing BounDArieS: eAi vS. etl
9.
the Bottom line
If data integration is the business pain point you
are facing, the most effective solution will be ETL.
However, if your real problem is process integration,
you will be better off with an EAI implementation.
SYNTEL: about
v i s i t S y n t e l ' s w e b s i t e a t w w w . s y n t e l i n c . c o m
SYNTEL
525 e. Big Beaver, third floor
troy, mi 48083
phone 248.619.3503
info@syntelinc.com
Syntel provides custom outsourcing solutions to global 2000
corporations. founded in 1980, Syntel's portfolio of services
includes Bpo, complex application development, management,
product engineering, and enterprise application integration
services, as well as e-Business development and integration,
wireless solutions, data warehousing, crm, and erp.
we maximize outsourcing investments through an onsite/off-
shore global Delivery Service, increasing the efficiency of how
complex projects are delivered. Syntel's global approach also
makes a significant and positive impact on speed-to-market,
budgets, and quality. we deploy a custom delivery model that
is a seamless extension of your organization to fit your business
goals and a proprietary knowledge transfer methodology to
guarantee knowledge continuity.