Y A W L

Chapter 19
Process Integration

Lachlan Aldred

Y
real world
R
a university for the © 2009, www.yawlfoundation.org
Overview Y

• Introduction

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 2
Introduction Y

• Process Integrations (PIs) need to couple (connect)

differently depending on the purpose of integrating.
– E.g. Feedback or Fire-and-forget
– Terms such synchronous/ asynchronous, point-to-point publish
subscribe refer to this coupling but have many meanings.

• Industry solutions create adapters in earnest. One of

the most compelling Process Integration products
claims over 350 adapters.
– No real support for aligning coupling with intent.
– It is as if the what (the thing being integrated with) has
drowned out the how (online/offline).

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 3
Introduction … Y

• Languages for process integration are not taken

seriously enough.
– Important efforts such as BPEL and BPMN are not given the
mindshare they deserve.
– Neither is formally defined.
– Neither can handle correlated event processing.

• This lecture will not solve process integration.

– Merely lay some conceptual groundwork.
– Rediscover and redefine some old ideas.
– Perhaps show some attributes we’ll see in future languages.

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 4
Process Integration as a Field of Endeavour Y

• Workflow Reference Model (WRM) defined 5 interfaces

– Interface 3 addresses invoking remote applications from a
process.
– Interface 4 addresses being integration with remote processes.

– Large effort - requires programming to these interfaces.

• BPM systems and ESBs superseded WRM.
– moved/reduced the need to program to APIs.
– Represented a major improvement for business users.
• BPEL enabled SOA paradigm applied to Business
Processes.
– Interfaces 3 & 4 of WRM became invoking a Web services and
exposing a process as a Web service – simple.
– Sending incoming requests to the right process instance was
addressed cleanly through Correlation-sets.
real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 5
Why do we Care? Y

• BPEL is a language for Process Integration so why

improve it?

• Expressive
– Can’t step outside of the Web service paradigm.
– Can’t handle correlated event processing.
– Models cannot discriminate between tightly coupled and
loosely coupled middleware (middleware is hidden in the
WSDL binding).

• Conceptualisation
– No publish-subscribe construct.
– Over the wire message format exposed to process layer.

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 6
Why do we Care …? Y

• BPEL is…

• Suitability
– Difficult to model batch message processing.
– Difficult to model message filtering.

• Understandable
– No graphical representation.
– One sided: can’t model two integrated processes together.

• Precise
– Rules of BPEL written in English : ambiguity, interpretation.

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 7
Process Integration is not Trivial Y

• Heterogeneity – of process engines, middleware,

applications (most challenging)
• Uncertainty/trust
– 2 Generals paradox
– Cannot trust that other general will co-attack based on
messages.
– There is no message protocol that can guarantee intent.
• Reliability
– Latency of messages (delays, stale data)
– Lost messages, duplicates deliveries.
• Events
– Don’t wait for them to occur – just handle it if it occurs.
– Composite events : Don’t do anything unless getX(Event_A) =
getX(Event_B)

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 8
Process Integration is not Trivial … Y

• Batch Messaging
– handle/create large numbers of messages.
– Avoid clumsy loops and lists

• Conversations
– Avoid clumsy correlation IDs.
– Correlation ID variables
– Technology agnostic
– Nested conversations

• Channel passing
– Processes learn of new message source/sinks through
message contents.
– Technology agnostic

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 9
Conceptualizing PI Y

• Conceptual modelling community have learned that

conceptualising a domain has pitfalls.

• Avoiding pitfalls - guiding principles:

– 100 percent principle:
• Every aspect of process integration should be describable.
• Full expressiveness. No need to express aspects of process
integration outside the language.

– Conceptualization principle:
• Only need to express relevant aspects.
• Avoid needing to express every low-level detail (e.g. access
credentials, message formats).

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 10
Conceptualizing PI … Y

• Software development community have learned that

abstracting a domain can fail:

• e.g. Law of Leaky abstractions:

– “all non-trivial abstractions, to some degree, are leaky“
(Spolsky).
– Abstract APIs can save huge amounts of development time,
but leaky ones cost just as much in learning time.

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 11
Why Patterns? Y

• Patterns express what the key static and dynamic

aspects of the domain are.
• Patterns, collectively, express what needs to be
modelled.
– Blueprint for a process integration language

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 12
Pattern Categories Y

• Patterns of Coupling
– Dimension of Threading, Time, and Space
• Batch Patterns
– Multiple message sends/receives, filtering.
• Bi-directional Interactions
– Various patterns for receiving feedback – within the interaction.
• Composed Interactions
– Patterns achieving state-aware conversations, across many
interactions.
• Event-based Process Patterns
– Patterns for handling unsolicited messages in-process

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 13
Pattern Categories … Y

• Transformations
– Patterns that transform messages and message data.
• Process Discovery
– Patterns that dynamically change during runtime to perform
new actions.

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 14
Coupling Pattern: Non-blocking receive Y

• Occurs when the receiving process is

not waiting for the message to arrive.
• Observable as foundation of most
asynchronous architectures.
• Enabler for Event-handling.

• e.g. reallocate seats for airline

cancellations.

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 15
Coupling Pattern: Time Decoupled Y

• No indication back to sender

about success of interaction.
• ‘Fire and forget’ – send message
and start doing something else
while the interaction is running.

• e.g. Reconciling inter-bank

payments overnight.

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 16
Composed Interaction: Property-based Y
Conversation
• Modelled by expressing functions over messaging tasks
• You only need to model the function logic and identify
the tasks involved.
• The process environment does the matching.
• When a match is found the message is directed to the
right process instance.

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 17
Conclusions Y

• Patterns have been used in software engineering to

express conceptual solutions to re-occurring problems.

• Process Integration Patterns, on the other hand, are

requirements patterns.
– They express what process integration needs to b able to do.

• They indicate what is relevant and what is less relevant

(Conceptualisation Principle).
• They indicate what a fully expressive PI language
should be capable of (100 Percent principle).

real
R
a university for the
© 2009, www.yawlfoundation.org
world Y A W L 18