Getting The Most Out of IIB and App Connect Enterprise (ACE) !

Getting the Most out of IIB and
App Connect Enterprise (ACE)!
Christopher Frank
IBM Cloud - Integration & Messaging
Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Agenda
¡ Where We Are
¡ What’s New in V11?
¡ Container Orchestration
¡ ACE and Containers
¡ ACE and MQ
¡ Summary

IBM Integration Bus
¡ Universal Connectivity FROM anywhere, TO anywhere
Simplify application connectivity for a flexible & dynamic infrastructure
¡ Comprehensive Protocols, Transports, Data Formats & Processing

Connect to applications, services, systems and devices
MQ, JMS 1.1, HTTP(S), SOAP, REST, File (incl. FTP, FTE, ConnectDirect), Database,
TCP/IP, MQTT, CICS, IMS, SAP, SEBL, .NET, PeopleSoft, JDEdwards, SCA, CORBA,
email…
Understand the broadest range of data formats
Binary (C/COBOL), XML, CSV, DFDL, JSON, Industry (SWIFT, EDI, HL7…), IDOCs,
user-defined
Built-in suite of request processors
Route, Filter, Transform, Enrich, Monitor, Publish, Decompose, Sequence, Correlate,
Detect…

Under the Covers
Node Instance
Node
Availability
Integration Integration Integration
(bipservice)
Server 1 Server 2 Server N
Administrative
Agent
(bipbroker)
Node wide
HTTP listener
(biphttplistener) (DataFlowEngine) (DataFlowEngine) (DataFlowEngine)
Node Queue Manager (amq*)

A Node Instance – Development and Operation
Development Operational
Integration
API Web UI Commands
(incl CMP)
MQ MQ
HTTP Node Instance HTTP
SOAP SOAP
JMS IS 1 IS 2 IS n JMS
Database Database
EIS… EIS…

What’s Changed with V11?
Copyright
Page
6 IBM 2018 MQ Technical Conference v2.0.1.8
“After”
How Things Were (V9/V10 and Before)
Flow
BAR ¡ Integration Node was the Control Point
Deployment, Admin, CLI, etc
Configurable Services & Policies
Admin Web UI
Process Supervision
Configuration Store
HTTP Listener
¡ Could (and often did) include Locally-Bound
HTTP Node Queue Manager
Listener MQ inbound/outbound message traffic
Global Transaction Coordinator
Used by some nodes to maintain state
Process (Collector. Timer, etc)
Supervision
¡ Usually Deployed on a Server or VM
Internal Configuration Store Due to size and complexity
Configurable Service Docker Containers supported since 2015
Physical Machine or VM

Flow Policy
BAR
What’s Changed in V11? Admin Web UI
¡ “Autonomous” Integration Servers

No Managing Integration Node required HTTP
Single OS Process Listener
Deployment, Admin, CLI, etc
”Public” Configuration Store (flows, policies, etc) Public Config Store
Container Flow Policy
¡ No Process Supervision
External Assumed (e.g. Orchestration Engine)
¡ Local Queue Manager still an option…

…But Not Recommended (clients preferred)
No Global Transaction Coordinator
Restrictions on some nodes (Collector, Timer, etc) How do I migrate to this???
¡ Redesigned for Deployment in a Container
Single process, reduced footprint

Flow Policy
BAR
“Before” Admin Web UI
Flow
BAR
HTTP
Admin Web UI Listener
Public Config Store
HTTP Listener Container Flow Policy
UNZIP and GO!
HTTP Node
Listener Flow Policy “Soon”
BAR
Process Admin Web UI
Supervision
Node
Internal Configuration Store
Configurable Service Process config
Supervision
Physical Machine or VM
UNZIP and GO!
Public Config Store
Flow Policy
Container Orchestration
Copyright
Page
Container Orchestration – What’s the Big Deal?

Orchestration Engines

1
4
Why Kubernetes?

Kubernetes Architecture

Node Instance
Node
Availability
Integration Integration Integration
(bipservice)
Server 1 Server 2 Server N
Administrative
Agent
(bipbroker)
Node wide
HTTP listener
(biphttplistener) (DataFlowEngine) (DataFlowEngine) (DataFlowEngine)


ACE and Containers
Copyright
Page
App Connect Enterprise – Integration Server Internals

Current Stateful Configuration of an Integration Server
Component Properties
(mqsichangeproperties)
BAR files Accounting & Statistics

.appzip (mqsichangeflowstats)
.libzip Resource Statistics
.shlibzip (mqsichangeresourcestats)
.msgflow Flow Monitoring
.cmf (mqsichangeflowmonitoring)
.xsd
.dfdl Configurable Services
Credentials
Work Load
(mqsisetdbparms)
Management
Policy

2
1
Architecture changes in App Connect Enterprise

> mqsideploy
> mqsicreateconfigurableservice
Message flow (Eclipse XML)
> mqsichangeproperties
> mqsichangeflowstats
BAR files
Node Policies
(bipbroker.exe)
In memory state Admin Services
Config Object
-Properties
-Description
Config Store -Links
Instruction Set XML
DataFlow Object
-Properties
-Description
Data Input -Links Data Output DataFlow Object
-Properties
Data Input -Description Data Output
-Links
Integration
Server Integration
Before Serialized XML

(dataflowengine.exe)
After Server
(dataflowengine.exe)

2
2
Welcome to your new friend - server.config.yaml

---
# ACE server configuration file
#
# General notes : Integration Server
# Integration Server will load server.conf.yaml from directory set via --work-dir
# File paths may be taken as absolute, or relative to the integration server's
work directory
serverConfVersion: 1 ResourceManagers:
adminRestApiPort: 7600 JVM:
defaultQueueManager: ''
httpConnectorPort: 0
Configuration! resourceStatsReportingOn: ''
jvmVerboseOption: none
httpsConnectorPort: 0 IntegrationServer: jvmDisableClassGC: ''
trace: none # set 1 of : none|service|diagnostic activeUserExitListi: '' jvmEnableIncGC: ''
traceNodeLevel: none inactiveUserExitList: '' jvmShareClasses: ''
StatsSnapNodeDataLevel: basic # choose 1 of : none|basic|advanced unnamedTraceLevel: none # choose 1 of : none|debugTrace jvmNativeStackSize: -1
StatsSnapOutputFormat: "csv,bluemix,usertrace" # comma separated list of : unnamedUserTraceLevel: none # choose 1 of : jvmJavaOSStackSize: -1
csv,bluemix,json,xml,usertrace none|debugTrace jvmMinHeapSize: 33554432
StatsSnapPublicationOn: false traceNodeLevel: true jvmMaxHeapSize: 268435456
StatsSnapThreadDataLevel: none # choose 1 of : none|basic processorArchitecture: '' jvmDebugPort: 19790
StatsSnapAccountingOrigin: '' consoleMode: '' jvmSystemProperty: ''
resourceStatsReportingOn: false httpNodesUseEmbeddedListener: '' keystoreType: ''
soapNodesUseEmbeddedListener: '' keystoreFile: ''
UserVariables: # equivalent to providing extra user variables on the command threadLocalProxyNameManagers: '' keystorePass: ''
line for IntegrationServer injectionMod: '' truststoreType: ''
Thing1: value failedMessageWaitTime: '' truststoreFile: ''
Thing2: value StatsSnapPublicationOn: '' truststorePass: ''
StatsSnapThreadDataLevel: '' crlFileList: ''
EnvironmentVariables: # equivalent to setting environment variables used StatsSnapNodeDataLeve: '' enableCRLDP: ''
during application source load and run StatsSnapOutputFormat: '' kerberosConfigFile: ''
Thing1: value StatsSnapAccountingOrigin: '' kerberosKeytabFile: ‘’
Thing2: value resourceStatsReportingOn: ''

2
3
Pet Cattle
Applications Worker Node

Virtual Machine Services Container
REST APIs
Code Code
Policies
Fixed Fixed
IntegrationServer flags Configuration
Configuration
server.config.yaml
App Connect Enterprise
Environment Runtime
Configuration Created new for each
new code version
App Connect Enterprise Remains same for each

new code version
Runtime
Host – including
Kernel
Environment
Configuration
HTTP Listeners in IIBv10 and ACEv11
IIBv10 ACEv11

ACE v11.0.0.1
Integration Node capabilities (Tech Preview)
ACE v11.0.0.0 Node-wide HTTP Listener (Tech Preview)
Web User Interface enhancements for support of Nodes
Unzip and Go - stand-alone Integration Servers Migration from IIBv9 and IIBv10
Policy creation in Toolkit & Deploy using a BAR Monitoring profiles
New Web User Interface Additional admin commands and extended REST APIv2
New REST APIv2 Web User Interface enhancements for support of Nodes
Server config yaml file for start up config New Group nodes for non-persistent in-memory aggregation
New App Connect Enterprise Developer edition
New Dockerfile @ github.com/ot4i/ace-docker
New Docker image @ hub.docker.com/r/ibmcom/ace/
New Cloud Connector Plan entitlement
Callable Flows across multi-tenant and dedicated runtimes ACE v11.0.0.3
ACE v11.0.0.2
Now
IIB v10.0.0.10
Q3 2017
Docker image on Docker Registry Hub Time
IIB Helm chart for running in Kubernetes
Send Resource Stats to Bluemix Logging
Send Resource Stats to Filesystem
Import Swagger with recursive references
Toolkit Export remote node connection info
New CPU benchmark processing command
IIB v10.0.0.14
IIB v10.0.0.11 IIB v10.0.0.12 IIB v10.0.0.13

Q4 2017 Q1 2018 Q2 2018
App Connect REST Request node Node.js v8.10.0 upgrade
Windows Server 2016 support
App Connect REST Pattern Support for Oracle 12c Release 2
Node.js v6 upgrade
Send Activity Log to IBM Cloud Support for DB2 version 12 on z/OS
Flow stack reporter
Ubuntu 16.04 support JD Edwards 9.2 support Support for Sybase v16
Copyright IBM 2018 Java 8 MQ Technical Conference v2.0.1.8
ACE, Containers and MQ
Copyright
Page
Docker Image Options for ACE and MQ
Docker Image Docker Image Docker Image
Integration Server Integration Server Integration Server

MQ Client Local Queue Manager
MQ connectivity Yes (HTTP API) Yes (client binding) Yes (server binding)
1PC for MQ No Yes Yes
EDA nodes No No Yes
2PC No No Yes
Horizontally scalable Yes Yes Yes

(with loss of sequencing) 27
Persistent volume Not required Not required Required

(if durability desired)
Start up Fast Fast Slower
Disk space Smallest Medium Largest

Same ACE/MQ image
…but different configuration and usage
ReplicaSet “replicas=n” StatefulSet “replicas=1”
Kubernetes Pod Kubernetes Pod
Docker Container Docker Container
Integration Server Integration Server
Local Queue Manager Local Queue Manager Persistent

Volume
replicas = n replicas = 1
no persistent volume claim persistent volume claim
• HA by replication (continuous availability) • HA by reinstatement
• Elastic horizontally scalability • Manual horizontal scalability
• Non-durable use of EDA nodes • Durable use of EDA nodes
• No 2 Phase Commit • 2 Phase Commit

Containers and pods for ACE and MQ
Pod Pod
Pod
Container
Container Container
Integration Server
Integration Server Integration Server
“server” “server”
connection connection
“client” (IPC) (IPC)
connection
Queue Manager Container
(TCP/IP)
Pod Queue Manager
Container • Requires combined ACE/MQ image –

not ideal from container design point of
Queue Manager view. Prefer to only have one core • Standard ACE and MQ images can
purpose/process per container. be used
• Enables event nodes • Enables event nodes
• Enables 2PC (MQ, ODBC, JMS, • Enables 2PC (MQ, ODBC)
• Standard ACE and MQ
JDBC, CICS) • Requires Docker 1.12 or later and
containers can be used
• Ties topology of ACE and MQ together. Kubernetes 1.10 (alpha feature)
• Does not enable event nodes
TCP/IP = Network based inter-communication
• Does not enable 2PC IPC = Inter Process Communication (via shared memory)

The Benefits of servers that don’t need a local queue manager
• SIZE: The size of the installation is dramatically reduced, and thereby the size of the Docker image. This reduces
build times due to the reduced image creation time, and deployment times as a smaller image is transported out to
the environments.
• MEMORY: The running container-based on the image uses significantly less memory usage as it has no processes
associated with the MQ server. Cloud infrastructure used for container-based deployment is often charged based on
memory rather than CPU so this can have a significant impact on running cost.
• START-UP: Start-up times of the containers are much faster as only one operating system process is started – that
of the integration engine. This improves agility by reducing test cycle time, and improves resilience and elastic
scalability by being able to introduce new runtimes into a cluster more rapidly.
• VOLUMES: MQ holds its message data on persistent volumes, and specific servers need access to specific
volumes within the MQ topology. If IBM App Connect Enterprise has a local MQ server, it becomes locked into
this topology. This makes it more complex to elastically add new servers to handle demand dynamically. For those
using Kubernetes it may result in a StatefulSet rather than the more straightforward ReplicaSet. Once again, this
makes it harder to take advantage of the cost benefits of elastic cloud infrastructure.

ACE Transactions and Queue Managers (1)

ACE Transactions and Queue Managers (2)

In Summary…
Copyright
Page
Summary
¡ Integration Architectures are Evolving
¡ ACE V11 is following this Evolution
¡ Container Orchestration
¡ ACE and Containers
¡ ACE and MQ

Questions & Answers

Getting The Most Out of IIB and App Connect Enterprise (ACE) !

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Getting The Most Out of IIB and App Connect Enterprise (ACE) !

Uploaded by

Copyright:

Available Formats

Getting the Most out of IIB and

App Connect Enterprise (ACE)!

IBM Cloud - Integration & Messaging

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

¡ What’s New in V11?

¡ ACE and Containers

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

¡ Comprehensive Protocols, Transports, Data Formats & Processing

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Node Queue Manager (amq*)

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

¡ “Autonomous” Integration Servers

¡ Local Queue Manager still an option…

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

BAR files Accounting & Statistics

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Architecture changes in App Connect Enterprise

Before Serialized XML

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Welcome to your new friend - server.config.yaml

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Applications Worker Node

App Connect Enterprise Remains same for each

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

IIB v10.0.0.11 IIB v10.0.0.12 IIB v10.0.0.13

Integration Server Integration Server Integration Server

1PC for MQ No Yes Yes

EDA nodes No No Yes

Horizontally scalable Yes Yes Yes

Persistent volume Not required Not required Required

Disk space Smallest Medium Largest

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Kubernetes Pod Kubernetes Pod

Docker Container Docker Container

Integration Server Integration Server

Local Queue Manager Local Queue Manager Persistent

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Container • Requires combined ACE/MQ image –

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

¡ ACE V11 is following this Evolution

¡ ACE and Containers

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

Copyright IBM 2018 MQ Technical Conference v2.0.1.8

You might also like