Adoption of XTP (eXtreme Transaction Processing)

leveraging grid-enabled Federated Enterprise Service Bus

(FESB)
Shivaji Sarkar
TATA Consultancy Services
Shivaji.Sarkar@tcs.com
Peter Mendis
TATA Consultancy Services
Peter.Mendis@tcs.com
ABSTRACT
Gartner predicts that By 2011, more than 20% of new TP application projects will require support for more than 10,000 concurrent
accesses or more than 500 transactions per second, up from less than 10% in 2006 (0.7 probability).
To avoid vertical scaling on expensive and proprietary hardware and software, eXtreme Transaction Processing applications are
architected to scale horizontally to minimize initial deployment costs and optimize the cost of scaling up. One such foundational
technology for XTP is Enterprise Service Bus (high-performance interaction with distributed resources) given the distributed, service-
oriented and event-driven nature of most new transactional applications.
However, the shortfall of scalability and performance even with the combination of Enterprise Application Servers (EAS) and ESB
technology to meet the requirement of the most extreme scenarios, forces enterprises to adopt incremental (Computing Appliances e.g.
Xi150) and emerging technologies (Enterprise Grid Architectures) without forcing substantial application redesign.
This article aims to establish the options of adopting the optimum solution for eXtreme Transaction Processing leveraging service
virtualization, federated enterprise service bus and grid-architecture.

1. INTRODUCTION
EXTREME TRANSACTION PROCESSING WHAT & WHY
Extreme Transaction Processing (XTP) is an exceptionally demanding form of transaction processing. Transactions of most high-end
(more than 10,000 concurrent accesses or 500 transaction per second) or ultra-high-end (more than 100,000 concurrent accesses or
5,000 transaction per second) requirements or more would require this form of processing.
Gartner defines XTP as an application style aimed at supporting the design, development, deployment, management and maintenance
of distributed TP applications characterized by exceptionally demanding performance, scalability, availability, security, manageability
and dependability requirements.
Very much like traditional TP systems, XTP applications are aimed at enabling efficient, reliable concurrent and real-time access
(read/update) to a shared database by executing application programs commonly referred to as "transactions."
The first and foremost adoption of XTP can be observed amongst the financial institutions (A 1-millisecond advantage in trading
applications can be worth $100 million a year to a major brokerage firm, by one estimate), whose prime requirements are more
processing capability, but without requiring exponential increase in hardware costs, in areas such as fraud detection, risk computation,
and stock trade resolution, to profit from minute, fleeting price anomalies and to mask their intentions via "time-slicing," or carving huge
orders into smaller batches so as not to move the market.

FEDERATED ESB (ENTERPRISE SERVICE BUS) - WHAT

Federated ESB (FESB) provides the ability to use multiple service registries and administration domains while mapping the disparate
registries into a federated set of services. The federated ESB facilitates service interactions with multiple ESB implementations. One
master ESB to which several dependent ESBs are federated to provide a subset of services that are applicable throughout the
enterprise.

Fig. Federated Enterprise Service Bus

GRID ARCHITECTURE - WHAT

Grid Architecture aim to integrate, virtualize, and manage resources and services within distributed, heterogeneous, dynamic virtual
organizations. The realization of this goal requires the disintegration of computing systems within and across organizations for
resources to be accessed as and when required, regardless of physical location.
The two key players in Grid environment are resource providers and resource consumers. Resource consumers adopt the strategy of
solving their problems at low cost within a required timeframe and resource providers adopt the strategy of obtaining best possible
return on their investment.
The Supply-Demand between Resource Providers and Resource Consumers are guided by the following key functional and non-
functional requirements:
i. Interoperability and Support for Dynamic and Heterogeneous Environments
ii. Optimization, Ease of Use and Extensibility
iii. Quality of Service (QOS) Assurance
iv. Security
v. Scalability & Availability
The utility provided by such Grid infrastructure is realized as a set of capabilities:
Request Management & Optimzation, Resource Management & Optimization, Infrastructure Optimization & Fedreration.

2. Challenges in Adoption of XTP

The fundamental challenge of XTP is supporting business-critical requirements comparable (or even more demanding) to those of
classic TP systems, but in a more distributed, interoperable, standards-based and global-class environment. XTP requirements are
fostering a new cycle of potentially disruptive innovation in application platforms involving:
Atomicity The effect of the execution of a transaction cannot be partial. A transaction must be fully executed or its partial effects must
be rolled back in case of failure.
Consistency Execution of a transaction cannot break the integrity rules associated with the database.
Isolation While a transaction is being executed, the data it is working on is not visible by other applications, so no other application
can have access to data in an intermediate state.
Durability Once a transaction has been completed (or "committed"), the corresponding database state change is persisted and
cannot be rolled back.
Availability Minimal planned system downtime for application or system upgrades
Reliability Fast and transparent recovery from hardware/software failures
Manageability Installation, configuration, administration, monitoring, maintenance and change management
Security Authentication, authorization, access control and communication encryption
Performance Execution of transactions within a pre-defined time frame (typically subseconds), irrespective of the number of users
concurrently accessing the system
Scalability Support to large and fluctuating workloads by optimally and dynamically allocating resources accordingly
Accountability Tracking and tracing operations for auditing or other management reasons
Extensibility Adding new transaction types rapidly without stopping operations
Faster Infrastructure Dedicated Compute Resources, Garbage Collection Pauses, Memory Heap Size, Lock Contentions Limit
Scalable Performance
Costs Utilization, Footprint, Power and Cooling

3. Leveraging Technology for XTP

From traditional TPMs and database management systems (DBMSs) to ESB technology of today, technology has evolved dramatically
to support high scalability and performance, optimized resource allocation, fast recovery and transaction integrity on top of the bare
operating system and file system. However, the combination of EAS and ESB technology cannot provide the scalability and
performance required by the most extreme scenarios without forcing substantial application redesign.

Fig. Technology options for XTP

Incremental Technologies
 Fig. Incremental Technologies

Emerging Technologies
 Fig. Emerging Technologies

4. Service Virtualization through Federated ESB in Grid-enabled environment

What is Service Virtualization?
Service Virtualization is the ability to hide operating system, platform technology and network connection or transport and helps the end-
user to work with simplified resources. In other words, one has to just know and implement business logic without any knowledge of
other underlying pieces. Polymorphism is best suited example for Code-level virtualization.
When implementing Business services there are four things that one needs to look at:
A. Business Logic
B. Deployment/Runtime Configuration
C. Security/Compliance Configuration
D. Messaging Infrastructure Details

Another aspect what one needs to know is the reuse and repeatable code that is churned out in Development lifecycle of a
product/project. In any given scenario during development there will be larger code reuse applicable from Deployment, Security and
Messaging standpoint. Such implementation shouldnt be duplicated anywhere in a typical Enterprise SOA Model. Instead in the above
scenario, the set of configuration has to be coded only one time at one place. So in any example if there are three services that are
needed to be developed and deployed separately, then what means by service virtualization is that only Business logic (A) need to be
written for disparate services while (B) (C) (D) needs to be just configured and reused.
Characteristics and Benefits of Service Virtualization:-
1) Enables factory development Model giving way for high Productivity.
2) Packaging & Container Deployment
3) Significant Code Reduction with reconfigure ability on Service Binding , Security and Policy
4) Minimal effort on Service Administration & Code Release.
5) Technology Independence.
So in any SOA Enterprise, what means by Service Virtualization is Developing Service with keeping Deployment and Services
Management to minimum and constant.

Service Virtualization through Federated ESB:

Enterprise Services Bus (ESB) is the backbone for any SOA enterprise. ESB helps Virtual zing the services. In a larger enterprise you
may need Multiple ESBs based on LOBs to rollout huge SOA initiatives. This is because of growing demand on Performance and SLAs.
Different ESBs emerge with different functions. These ESBs might be similar or different based on one or multiple vendor selection.
In todays ESB's evolution we see that ESB has come miles from Foundational to Transformational and now with one that can adapt
dynamically called as Federated ESB (FESB). (Pronounced as fis-bee)
In Transformational environment ESB would use Service Registry, do BPM Service Orchestration, Service Activity and Systems
Monitoring with or without using Computing Appliance Systems.
Federated ESB (FESB) would not only use above features but also have services hosted on multiple ESBs. FESB may be owned by
multiple LOBs. They are managed separately but they work together as Single Golden Source for Enterprise Services. These FESB
need not proliferate from same vendor.
Dependencies:-
Single point of source for Registry Source.
Autonomous Manager managing and mediating service across FESB.

How Grid Environments enables optimization of Service Virtualization in Federated ESB?

FESB enabled by Grid enables high end performance based dynamically configurable environment.
In a Federated ESB Grid Environment, the Provider and Consumer had to scale and reconfigure based on runtime demand vs. supply
footprint. The Autonomous Manager would start or stop services across ESB. Know the health status of the services on distributed Grid
Environment across Federated ESB environments. In other words the Services Deployed in Federated ESB Grid Environment should
behave in manner of self implied Artificial Intelligence System.
Here are the bases for any SOA infrastructure to attain maturity on Federated ESB Grid:
1. Self Managed Autonomous Domain
2. Event Driven Application
3. Service Mediation
4. Heterogeneous Platform Support

Self Managed Autonomous Domain:

In Federated ESB Grid environment, there will be one Master ESB and several other Slave ESBs. Both master & slave will be workers
except that the Master will also manage the Services deployed in other ESB for throttling (Slow Consumer/ Fast Provider) or Service
Recovery.
At the moment of Master ESB failure, one of the Slaves will be promoted as Master and would start Managing until the Original Master
comes back online. This entire setup is called as Self Managed Autonomous Domain setup.
In an extreme Transaction Procession environment this setup would ensure high system performance since the Autonomous domain
would manage the services in such a manner that the least used ESB would be leveraged by scaling the services on it.
Event Driven Application
Event Driven Application Platforms support event centric programming models such as event handlers which are triggered by process
business events. The Service application that is exposed as part of ESB should support Event-Driven-Application (EDA) models.
Service Mediation:
The ESB should able to intercept and modify service messages that are passed between the services. It should process message
based on primitive or complex logic. It should able to transform, route or augment message within the grid environment.
Heterogeneous Platform Support:
Services that are hosed in the Grid enabled architecture should not be technology agnostic. It should be able to support JAVA, .net,
python, etc.

What Services need to be optimized?

Identification of services for optimization would be based on most used ones in terms of number of hits or services that conform to SLA
commitments. These services would have to be configured and deployed on various ESBs. At runtime these services would be
dynamically scale based on the policies written within the grid environment. Services that need to process extreme transaction would
leverage this underlying grid enabled FESB environment.

Fig. Federated ESB (FESB) in Grid Enabled Environment

5. CONCLUSION
XTP is a complex problem and cannot be solved with one specific product or technology. Organizations should follow an incremental
approach by complementing established and proven products with the appropriate combination of innovative technologies as needed for
supporting emerging business requirements or adopt the riskier and more-disruptive technologies to grab greater business benefits and
sustainable competitive advantage e.g. the combination of proven and advanced technologies is, in many cases, the most pragmatic
way to take advantage of innovation by keeping risk at a tolerable level.

Fig. Roadmap for Hybrid Technology Adoption

6. REFERENCES
[1] Gartner RAS Core Research Note G00146107, G00143682, G00131036
[2] Wall Street's Quest To Process Data At The Speed Of Light By Richard Martin
[3] Open Grid Services Architecture (OGSA) [Copyright Global Grid Forum (2002-2005)]
[4] A Case for Economy Grid Architecture for Service Oriented Grid Computing By Rajkumar Buyya, David Abramson, and Jonathan Giddy
[5] Service Virtualization:The Road to Simplification by Art Sedigh
[6] Enterprise Service Bus Roy Schutte.