Enterprise Integration

1
Agenda

 Challenges in enterprise application development

 Design patterns

 Enterprise Integration patterns

 Interop technologies

 Conclusion & Resources

2 © Copyright ThoughtWorks, Inc.® 2005

The challenge

 Enterprises depend on a growing number of IT systems

 The systems must provide an integrated solution for the

enterprise
 The systems must interoperate with each other

 Architectural trends and “waves” of technologies

 Changing business needs and requirements

3 © Copyright ThoughtWorks, Inc.® 2005

Finance Operator Application
fax/telex

app web

email

mail document
New System recognition

Existing Systems ERP ref. data

 Workflow items are stored on Tibco RV/CM queue

 Tibco Integration Manager
 organises workflow
 handles imports from document recognition
 handles interaction with existing ERP systems
 Reference data imported using linked servers
4 © Copyright ThoughtWorks, Inc.® 2005
Portfolio Management Tool
New application  Front-end uses Office 2003
Excel code-behind technology
front-end
 communicates with server using
WebSphere MQ queues
application
logic  Message format defined with
XSD schema
technology
adaptor  Application uses existing services
 accesses analytics library with
analytics deal messaging
COM bridge
library manager library  accesses stored deals through
deal manager service
 accesses service bus through
Existing Existing Existing messaging abstraction library

5 © Copyright ThoughtWorks, Inc.® 2005

Design Patterns

6
Why design patterns?

 Code reuse remains difficult

but…

 Knowledge reuse can be very valuable

 Patterns encapsulate knowledge of successful designs

7 © Copyright ThoughtWorks, Inc.® 2005

Using technology successfully
 Syntax
 Basic language mechanism
 A necessity but not a guarantee
 Constructs
 Classes, Interfaces, Inheritance, Polymorphism, etc.
 Helpful but not sufficient for good design
 Principles
 Separation of concerns, Dependency Injection, etc.
 Steer us towards a better solution
 Patterns
 Model-View-Controller, Observer, Decorator
 Concrete design guidance
8 © Copyright ThoughtWorks, Inc.® 2005
Patterns

 “Mind sized” chunk of information (Ward Cunningham)

 Shows a good solution to a common problem within a

specific context
 Observed from actual experience

 Has a distinct name

 Not copy-paste

 ThoughtWorks collaborates with Microsoft to document

design patterns for the .NET platform

9 © Copyright ThoughtWorks, Inc.® 2005

Enterprise Integration Patterns

Message Message Message

Construction Routing Transformation

Endpoint Endpoint
Message Router Translator
Channel
Application Application
A B

Messaging
Messaging Channels Monitoring System
Endpoints Management

 Gregor Hohpe defined a visual pattern language describing

message-based enterprise integration solutions
 Pattern language comprises 65 patterns in 6 categories

11 © Copyright ThoughtWorks, Inc.® 2005

Pattern: Request-Response

Consumer Request Provider

Request Channel

Reply Channel
Response

 Service Consumer and Provider (similar to RPC)

 Channels are unidirectional
 Two asynchronous point-to-point channels
 Separate request and response messages

13 © Copyright ThoughtWorks, Inc.® 2005

Multiple consumers

Consumer 1 Requests Requests Provider

Request Channel

Reply Channel 1 ??
Reply Channel 2
Responses
Consumer 2

 Each consumer has its own reply queue

 But how does the provider send the response?
 Could send to all consumers (very inefficient)
 Hard code (violates principle of context-free service)

14 © Copyright ThoughtWorks, Inc.® 2005

Pattern: Return Address

Reply Reply
Consumer 1 Channel 1 Channel 2 Provider
Request Channel

Reply Channel 1
Responses
Reply Channel 2

Consumer 2

 Consumer specifies Return Address (the reply channel)

in the request message
 Service provider sends response message to specified
channel

15 © Copyright ThoughtWorks, Inc.® 2005

Load-balanced service providers

Provider 1
Consumer

Request Channel

Provider 2

Reply Channel

 Request message can be handled by multiple providers

 Point-to-point channel supports competing services
 Only one service receives each request message
 But what if the response messages are out of order?

16 © Copyright ThoughtWorks, Inc.® 2005

Pattern: Correlation Identifier

Provider 1
Consumer Message
Identifier 1
1 2
1 2
Request Channel
2

Provider 2
2 1
2 1 2 1
Reply Channel Correlation
Identifier

 Consumer assigns a unique identifier to each message

 Identifier can be an arbitrary ID, a GUID, a business key
 Provider copies the ID to the response message
 Consumer can match request and response

17 © Copyright ThoughtWorks, Inc.® 2005

Multiple specialised providers

Order Messages Widget Inv.

Order
Entry
??
Gadget Inv.

 Each provider can only handle a specific type of message

 Route the request to the "appropriate" provider. But how?
 Do not want to burden sender with decision
 Letting providers "pick out" messages requires coordination

19 © Copyright ThoughtWorks, Inc.® 2005

Pattern: Content-Based Router

Order Messages Widget Inv.

Order
Entry
Gadget Inv.
Content
Based
Router

 Insert a content-based router

 Routers forward incoming messages to different channels
 Message content not changed
 Mostly stateless, but can be stateful, e.g. de-duper

20 © Copyright ThoughtWorks, Inc.® 2005

Composite messages

Order
Message
Widget Inv.
Order
Entry
??
Gadget Inv.

 How can we process a message that contains multiple

elements?

21 © Copyright ThoughtWorks, Inc.® 2005

Pattern: Splitter & Router

Order
Order Item 1
Order
Message Items Widget Inv.
Order
Entry
Gadget Inv.
Splitter Router
Order
Item 2

 Use a splitter to break out the composite message into a

series of individual messages
 Then use a router to route the individual messages as before

 Note that two patterns are composed

22 © Copyright ThoughtWorks, Inc.® 2005
Producing a single response

Order
Item 1 Response 1 Confirmed
Order
Widget Inv.
?? Billing
Gadget Inv.
Order Response 2
Item 2

 How to combine the results of individual but related

messages?
 Messages can be out-of-order, delayed
 Multiple conversations can be intermixed

23 © Copyright ThoughtWorks, Inc.® 2005

Pattern: Aggregator

Order
Item 1 Response 1 Confirmed
Order
Widget Inv.
Billing
Gadget Inv. Aggregator
Order Response 2
Item 2

 Use a stateful filter, an Aggregator

 Collects and stores messages until a complete set has been
received (completeness condition)
 Publishes a single message created from the individual
messages (aggregation algorithm)
24 © Copyright ThoughtWorks, Inc.® 2005
Communicating with multiple parties

Quotes
Vendor A

Vendor B
Request
For Quote
?? Vendor C

Best
Aggregator
Quote

 How to send a message to a dynamic set of recipients?

 And return a single response message?

25 © Copyright ThoughtWorks, Inc.® 2005

Pattern: Scatter-Gather

Pub-Sub Quotes
channel Vendor A

Vendor B
Request
For Quote
Vendor C

Best
Aggregator
Quote

 Send message to a pub-sub channel

 Interested recipients subscribe to a "topic"
 Aggregator collects individual response messages
 may not wait for all quotes, only returns one quote

26 © Copyright ThoughtWorks, Inc.® 2005

Complex composition

Pub-Sub Quotes
channel Vendor A

Vendor B
Splitter Quote request
for each item Vendor C

Best Quote
Aggregator Aggregator
for each item

 Receive an order message

 Use splitter to create one message per item
 Send to scatter/gather which returns "best quote" message
 Aggregate to create quoted order message

27 © Copyright ThoughtWorks, Inc.® 2005

Interop technologies

28
Major interop technologies

 Resource based
DB files

 RPC style / distributed objects

RMI-IIOP Remoting in-process
WS
(CORBA) (DCOM) bridge

 Message based / document-oriented

MOM WS WS-*

29 © Copyright ThoughtWorks, Inc.® 2005

Interop technology characteristics

platform J2EE .NET

in-proc
neutral server server

RMI-IIOP Remoting
DB/files MOM WS-* WS bridge
(CORBA) (DCOM)

point-to-point
routable

transient messages
durable message

clustering

server lifetime-
management

30 © Copyright ThoughtWorks, Inc.® 2005

Messaging

 Channels are separate from applications

 Removes location dependencies

 Channels are asynchronous & reliable

 Removes temporal dependencies

 Data is exchanged in self-contained messages

 Removes data format dependencies

 Loosely coupled integration enables independent variation

31 © Copyright ThoughtWorks, Inc.® 2005

Why Web Services?

 Web services provide all benefits of messaging solution

 loose-coupling
 service oriented
 reliable communication

 Why web services?

 composable protocol
 vendor neutral
 suitable for access through Internet

32 © Copyright ThoughtWorks, Inc.® 2005

Web Services development
 Web Services are expected to become the default Messaging
solution in the future

 WS-* standards are evolving rapidly, most important

 WS-Security
 WS-Policy
 WS-Addressing

 WS-I defines profiles, sample applications and testing tools

 Profiles are guidelines for using WS specifications
 Use Basic profile 1.1 to build interoperable solutions

 Further research on WS is being carried out

33 © Copyright ThoughtWorks, Inc.® 2005
Conclusion

 Enterprise systems are becoming more complex

 We have patterns to help us with the design

 We have technologies to implement our designs

 Building for interoperability and integration is key

35 © Copyright ThoughtWorks, Inc.® 2005

Recommended books
Enterprise Integration Patterns

Gregor Hohpe, Bobby Woolf

Addison-Wesley, 2004 Integration Patterns
Microsoft Patterns & Practices
2004

Patterns of Enterprise
Application Architecture
Martin Fowler
Addison-Wesley, 2003
Enterprise Solution Patterns
using Microsoft .NET
Microsoft Patterns & Practices
2003
Enterprise SOA
Dirk Krafzig, Karl Banke,
Dirk Slama
Prentice Hall, 2004

36 © Copyright ThoughtWorks, Inc.® 2005