TAFJ C a c h e

R17
 TAFJ C a c h e

Amendment History:

Revisio
Date Amended Name Description
n
1 1st June 2016 R. Vincent Initial version
2 11th July 2016 R. Vincent Add JBoss Data Grid
3 23rd July 2016 R. Vincent Add WAS Extreme Scale

4 14th Sept. 2016 R. Vincent Add BATCH_WRITE documentation

5 30th Jan. 2017 R. Vincent Add Apache Ignite

6 24th March 2017 R. Vincent Add Ignite Security section

Page 2
TAFJ C a c h e

Copyri g h t
Copyright (c) 2014 TEMENOS HOLDINGS NV
All rights reserved.
This document contains proprietary information that is protected by copyright. No part of this document may
be reproduced, transmitted, or made available directly or indirectly to a third party without the express
written agreement of TEMENOS UK Limited. Receipt of this material directly TEMENOS UK Limited
constitutes its express permission to copy. Permission to use or copy this document expressly excludes
modifying it for any purpose, or using it to create a derivative therefrom.

Errat a and Com m e n t s

If you have any comments regarding this manual or wish to report any errors in the
documentation, please document them and send them to the address below:
Technology Department

Temenos Headquarters SA
2 Rue de l’Ecole-de-Chimie,
CH - 1205 Geneva,
Switzerland

Tel SB: +41 (0) 22 708 1150

Fax: +41 (0) 22 708 1160

Please include your name, company, address, and telephone and fax numbers, and email
address if applicable. TAFJdev@temenos.com

Page 3
TAFJ C a c h e

Table of Contents
Copyright................................................................................................................................................ 3
Errata and Comments............................................................................................................................ 3
T24 Overall Infrastructure....................................................................................................................... 6
Prerequisites........................................................................................................................................... 7
Benefits of TAFJ Caching....................................................................................................................... 7
Setting up TAFJ Caching........................................................................................................................ 7
Common Setup................................................................................................................... 7
Property Setup................................................................................................................. 7
Caching.properties........................................................................................................... 8
Example of setting up your own cache.............................................................................9
Table Setup................................................................................................................... 10
Logging TAFJ................................................................................................................. 10
Batch Caching...................................................................................................................................... 11
MemoryGridLockManager.................................................................................................................... 11
Apache Ignite Setup............................................................................................................................. 11
Oracle Weblogic Coherence Setup...................................................................................................... 12
Oracle Coherence Setup for Clients..................................................................................14
Standalone Clients.........................................................................................................14
All Clients....................................................................................................................... 14
Logging Coherence...........................................................................................................16
Deploying the TAFJ.gar file...............................................................................................16
Running in Standalone Mode with Coherence...................................................................16
Monitoring Coherence.......................................................................................................18
JBoss Data Grid Setup......................................................................................................................... 18
Using with a Standalone client...........................................................................................19
Websphere Extreme Scale Grid Setup................................................................................................. 20

Page 4
TAFJ C a c h e

............................................................................................................................................................. 20
Modifying the ExtremeScale example to add some TAFJ Caches.....................................22
Starting your Catalog server..............................................................................................23
Starting a Container........................................................................................................... 23
Setup a Catalog Service Domain in WAS..........................................................................23
Standalone Extreme Scale................................................................................................25
Using TAFJCache as a Component..................................................................................................... 25
Monitoring Caching............................................................................................................................... 26
Appendix.............................................................................................................................................. 26
Estimating table sizes........................................................................................................26

Page 5
TAFJ C a c h e

T24 Overall Infra s t r u c t u r e

This step by step process will tell you how to setup caching for TAFJ.

Page 6
TAFJ C a c h e

Prer e q u i s i t e s
This assumes you have installed an external caching solution such as Oracle Coherence.
Henceforth, $COHERENCE_HOME will refer to the Coherence install directory.

Be n e f i t s of TAFJ Cach i n g
The benefits of TAFJ Caching are in execution time which vastly improves. One can see this
in the TAFJ Monitor where cache read time is tracked. As of this writing, TAFJ Caching
caches a set of defined (mostly) read-only tables which are defined in the TAFJ_CACHE
table. After a record is read from the database, the record is cached as a dynamic array (no
matter which cache mechanism is used). Subsequent reads go to the cache first to check it
if it is there. If it is, this is called a cache hit. If it isn’t, it is called a cache miss.

The record is removed from the cache before any write occurs so that any subsequent read
will get fresh data from the database. Thus it is better to configure tables which are written
infrequently to avoid cache misses which penalize performance.

Write-through cache might be implemented at a later date, but in this first phase it is not.

Se t t i n g up TAFJ Cac hi n g

Com m o n Set u p

Prop e r t y Set u p
In your TAFJ properties file, set the following property to enable external caching:

temn.tafj.runtime.use.external.caching = true

To enable batch write caching, set the following property to true:

temn.tafj.runtime.use.batch.write=true

Batch write caching caches records at the application until either a

TRANSEND/TRANSABORT or the batch size defined is reached, at which point the batch is
flushed to the database (TRANSEND or batch filled), or emptied (TRANSABORT). Batch
write caching is independent of external caching so one not need set
temn.tafj.runtime.use.external.caching to true as well. The amount of records to batch is set
at the thread level. See “Batch Caching”.

To enable dictionary caching, set the following property to true:

temn.tafj.runtime.use.cache.dict.item=true

Dictionary caching, as of now, is relegated to the TAFJ local caching mechanism as the
Object is a complex object and not a String. Local caching provides the fastest access.
Dictionaries are rarely, if at all updated. The whole dictionary cache expires occasionally and

Page 7
TAFJ C a c h e

is controlled by the temn.tafj.cache.dict.expiry.time property which is in milliseconds. The

default is 4 hours. The cache is also cleared if STANDARD.SELECTION is rebuilt, but this
would happen only on one node and is not cluster wide.

Cach i n g . p r o p e r t i e s
In $TAFJ_HOME/conf/cache directory, there is a caching.properties file which handles
caching properties. The reason this is separated from tafj.properties is that the
TAFJCaching.jar can be used as a separate component independent of TAFJ. (It is only
reliant on TAFJLogging.jar)

The property below allows you to switch a caching provider. The default is Coherence. A list
of supported providers appear in this properties file shown below.

temn.tafj.cache.external.caching.provider=com.tangosol.coherence.jcache.CoherenceBased
CachingProvider

# Provider of external caching services. Default is Oracle Coherence. See

$TAFJ_HOME/doc/TAFJCache.pdf to set up correctly for different providers

# Providers: Coherence use =>

com.tangosol.coherence.jcache.CoherenceBasedCachingProvider

# Providers: Jboss use => org.infinispan.jcache.embedded.JCachingProvider

# Providers: Websphere use => com.ibm.websphere.ObjectGridProvider

# Providers: Temenos (local cache only across one jvm) use =>
com.temenos.tafj.cache.TAFJLocalCacheFactory

# Providers: Temenos (replicated across multiple jvms) use =>

com.temenos.tafj.cache.TAFJReplicatedCacheFactory

# TAFJ properties will override this one if provided in tafj.properties. If using the API, this can
also be passed as a CacheParameter which takes ultimate precedence.

# Jboss Data Grid Server host and port

temn.tafj.cache.jboss.jdg.host = localhost

temn.tafj.cache.jboss.jdg.port = 11322

# Websphere Object Grid Server host and port

temn.tafj.cache.was.og.host.port = localhost:2809

Page 8
TAFJ C a c h e

temn.tafj.cache.was.og.grid.name = TemenosGrid

temn.tafj.cache.was.og.domain.name = DefaultDomain

# Reserved cache names

temn.tafj.cache.name.dict=DICTCACHE

temn.tafj.cache.name.write=WRITECACHE

TEME N O S Sp e c i f i c Pro p e r t y Se t u p
For Temenos caching which is a local cache spanning the current JVM only, you need to
setup the provider as below. (Note that TAFJRemoteCacheFactory supporting multi-JVM will
be supported in the near future).

temn.tafj.cache.external.caching.provider= com.temenos.tafj.cache.TAFJLocalCacheFactory

In this mode, caches are defined declaratively. By default, a Tables cache is set up in the
properties file as shown below (when creating the table TAFJ_CACHE and inserting the
default entries, the cache defined is called “Tables”). This cache expires entries after 4
hours (14400 seconds) or when the memory exceeds 100 megabytes for the cache. In this
case a “least recently used” policy would kick in booting out entries. A LFU (least frequently
used) policy is also supported.

# Configurable caches below. Configure you own by adding properties

temn.tafj.cache.name.<<My new cache>>=MyName, temn.tafj.cache.name.<<My new
cache>>.memory=2M, etc.

temn.tafj.cache.name.Tables=Tables

temn.tafj.cache.name.Tables.expiryTime.seconds=14400

temn.tafj.cache.name.Tables.memory=100M

temn.tafj.cache.name.Tables.evictionPolicy=LRU

Exa m p l e of set t i n g up your own cac h e .

Let’s call our cache SHORTLIVED and assign F.DATES table. Our entry in the
caching.properties file might look something like this:

temn.tafj.cache.name. SHORTLIVED = SHORTLIVED

temn.tafj.cache.name. SHORTLIVED.expiryTime.seconds=60

Page 9
TAFJ C a c h e

temn.tafj.cache.name. SHORTLIVED.memory=100M

temn.tafj.cache.name. SHORTLIVED.evictionPolicy=LRU

We would then insert a record into the TAFJ_CACHE for this, ie.

SQL> INSERT INTO TAFJ_CACHE(RECID,CACHENAME,WRITEBATCHROWS) VALUES

('F.DATES','SHORTLIVED',NULL);

Now whenever a row is read from the database, it will be cached locally across one or many
application servers (depending on the TAFJ caching provider used) for 60 seconds or until a
LRU policy evicted it.

JBos s Sp e c i f i c Pro p e r t y Se t u p
In the caching.properties file, for Jboss Data Grid you also need to setup the host and port.

temn.tafj.cache.external.caching.provider=org.infinispan.jcache.embedded.JCachingProvider

temn.tafj.cache.jboss.jdg.host=localhost

temn.tafj.cache.jboss.jdg.port=11322

(The above assumes Jboss Data Grid started with 100 port offset, ie. If you start with
.\clustered.bat -Djboss.socket.binding.port-offset=100, you should see:

09:24:03,773 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) JDGS010001:

HotRodServer listening on 127.0.0.1:11322

on the console)

W e b s p h e r e Sp e c i f i c Pro p e r t y Se t u p
In the caching.properties file, for Websphere Extreme Scale you also need to setup the host,
port, and grid name. The configured grid name on connection will use the cache name plus
the value in temn.tafj.cache.was.og.grid.name to connect. For example, if the cache name is
Tables then the object grid name to be connected to (and must match what is in
objectgrid.xml) is TablesGrid, ie

<objectGrid name="TablesGrid" txTimeout="60"> tag in objectgrid.xml.

temn.tafj. cache.external.caching.provider=com.ibm.websphere.ObjectGridProvider

temn.tafj.cache.was.og.host.port=localhost:2809, myremotehost:2809

temn.tafj.cache.was.og.grid.name=Grid

temn.tafj.cache.was.og.domain.name=DefaultDomain

Page 10
TAFJ C a c h e

Tabl e Set u p
A table is targeted for caching by the TAFJ_CACHE table. Configuration of tables is done in
the TAFJ_CACHE table. If this table doesn’t exist, create it by running the script
createtable_tafj_cache.sql in your %TAFJ_HOME%\dbscripts directory for the appropriate
database. This script will also insert some default tables as well so be sure to look and see
which tables are included. The current table structure is as follows:

SQL> desc tafj_cache;

Name Null? Type

----------------------------------------- -------- ----------------------------

RECID NOT NULL VARCHAR2(255)

CACHENAME VARCHAR2(255)

WRITEBATCHROWS VARCHAR2(5)

Once the table is created, create the appropriate rows for tables that should be cached. An
example follows:

INSERT INTO TAFJ_CACHE (RECID,CACHENAME,WRITEBATCHROWS) VALUES

('F.EB.LOOKUP','Tables',NULL);

For new DBImports using the DBImport progam, the TAFJ_CACHE table will be created for
you empty.

Log g i n g TAFJ
Control log level of TAFJ Caching logging with the following appender in
TAFJTrace.properties. Logs are written to %TAFJ_HOME%\log\cache.log

log4j.logger.CACHE=INFO, cache

Bat c h Cac hi n g
As mentioned previously, batch write caching is independent of external caching. To set it
up, first set your property to true.

temn.tafj.runtime.use.batch.write=true

Second, create a batch enabled file (example below):

CREATE-FILE BATCH.WRITE TYPE=XML BATCH_WRITE=8

What this does is 1) creates the file of type XML 2) creates a record in the TAFJ_CACHE
table with RECID= BATCH.WRITE, CACHENAME=WRITECACHE, and
WRITEBATCHROWS=8. WRITEBATCHROWS tells how frequently to flush to the database

Page 11
TAFJ C a c h e

once the cache is full PER THREAD. This is very important as you must size how much
memory this will take for all tables batched and how many threads will be running. Without
adequate memory, one could encounter memory violations.

So in the case above, JBC WRITE statements on the table BATCH.WRITE will cache rows
up to 8. On the 9th WRITE, records would then be flushed to the DB and the internal cache
would be emptied. The records would not yet be committed until a TRANSEND or
TRANSABORT are seen.

One can also set up existing tables by inserting records into the TAFJ_CACHE tables with a
RECID that matches the TAFJ_VOC RECID, a cache name of WRITECACHE, and some
valid integer for BATCH_WRITE.

Me m o r y Gri dL o c k M a n a g e r
The MemoryGridLockManager is a new lock manager that locks data using a caching
mechanism. DO NOT CHANGE IT AWAY FROM USING A REPLICATED SCHEMA.
DOING SO COULD HAVE TERRIBLE REPROCUSSIONS FOR YOUR APPLICATION
SHOULD A NODE IN THE CLUSTER GO DOWN.

To enable it, set the following property:

temn.tafj.locking.mode= MEMORYGRID

Currently, the Memory Grid Lock Manager is supported for Coherence, Jboss Data Grid
(using clustered.xml), and Websphere ExtremeScale.

Apac h e Igni t e Set u p

The Ignite configuration file can be specified with the temn.tafj.cache.ignite.conf.file.loc
property in caching.properties. For example,

temn.tafj.cache.ignite.conf.file.loc=<tafj.home>/conf/cache/ignite-cache.xml

For information regarding Ignite caching, including downloading and setup, please see the
Apache Ignite documentation at https://ignite.apache.org/index.html. Once you install Ignite
somewhere, this becomes the IGNITE_HOME.

In order to use Ignite with Jboss and other application servers, one must correctly set the
CLASSPATH to the Ignite libraries. To do this for JBoss, simply copy the following Ignite jars
to the %TAFJ_HOME%\ext directory. (This assumes you have a symbolic link to this
directory in the %JBOSS_HOME%\modules\com\temenos\tafj directory, otherwise you need
to copy each jar in the modules directory). Jars to include:

%IGNITE_HOME%\libs\ignite-core-1.8.0.jar,

%IGNITE_HOME%\libs\ignite-shmem-1.0.0.jar

%IGNITE_HOME%\libs\ignite-spring\*.jar (all jars under this directory)

Page 12
TAFJ C a c h e

Now reference them in module.xml like the following.

<resource-root path="/ext/ignite-core-1.8.0.jar"/>

<resource-root path="/ext/ignite-shmem-1.0.0.jar"/>

<resource-root path="/ext/ignite-spring/ignite-spring-1.8.0.jar"/>

<resource-root path="/ext/ignite-spring/commons-logging-1.1.1.jar"/>

<resource-root path="/ext/ignite-spring/spring-aop-4.1.0.RELEASE.jar"/>

<resource-root path="/ext/ignite-spring/spring-beans-4.1.0.RELEASE.jar"/>

<resource-root path="/ext/ignite-spring/spring-context-4.1.0.RELEASE.jar"/>

<resource-root path="/ext/ignite-spring/spring-core-4.1.0.RELEASE.jar"/>

<resource-root path="/ext/ignite-spring/spring-expression-4.1.0.RELEASE.jar"/>

<resource-root path="/ext/ignite-spring/spring-jdbc-4.1.0.RELEASE.jar"/>

<resource-root path="/ext/ignite-spring/spring-tx-4.1.0.RELEASE.jar"/>

<resource-root path="/ext/ignite-spring/spring-tx-4.1.0.RELEASE.jar"/>

<resource-root path="/ext/ignite-indexing/ignite-indexing-1.8.0.jar"/>

etc

In addition, because of an error that occurs you must include the following entry under
dependencies.

<dependencies>

<module name="sun.jdk" export="true" >

<imports>

<include path="sun/misc/Unsafe" />

</imports>

</module>

</dependencies>

Likewise, if you are using Weblogic or Websphere, set the appropriate jars to the
CLASSPATH.

Page 13
TAFJ C a c h e

Igni t e Se c u r i t y
Apache Ignite does not ship with security by default, so Temenos has added its own. Nodes
joining the cluster are authenticated by 1) ip address (independent of Apache Ignite’s IP
checking) and 2) a password stored in a java key store.

IP Ch e c k i n g
A node starting will be authenticated by the property temn.tafj.cache.ignite.allowed.ips. The
node starting must be an ip recognized in one of these properties. Ignite itself has its own ip
checking, but this is added security.

temn.tafj.cache.ignite.allowed.ips=127.0.0.1,10.41.5.51,10.30.201.46

Pas s w o r d Aut h e n t i c a t i o n
The Ignite password is stored within the keystore file %TAFJ_HOME
%\conf\cache\cachestore.keystore file. The initial password is encrypted on disk in the
%TAFJ_HOME%\conf\cache\caching.properties file and compared against the password in
the keystore when a node starts up. The keystore password is also encrypted and stored in
the caching.properties file like the below.

temn.tafj.cache.keystore.password=RxtJqo9ToOblM8T0svl4KQ==

temn.tafj.cache.ignite.password=6fiHGU8oaQ+F8KdDw9v5jQ==

The caching framework decrypts the keystore password by using an initializer. This initializer
is first attempted to be retrieved from the database. The reason for so doing is that the
database has the ability to encrypt tables and tablespaces to provide another layer of
security. To find out how to encrypt a table please see your specific database
documentation. If no RECID is found in the TAFJ_CACHE_KS_INIT table where the
initializer exists, the default initializer is used from the %TAFJ_HOME%\conf\cache\ignite-
cache.xml file. The caching framework tries to speak with the database through the T24
datasource. If you are in an application server context, the following caching properties will
be used:

temn.tafj.jdbc.url=

temn.tafj.jdbc.driver=

temn.tafj.jdbc.username=

temn.tafj.jdbc.password=

Again, the above properties are only used in a standalone mode where you are outside the
application server.

Cac h e Enc r y p t i o n
Data String values (not keys) in all of the Ignite caches are encrypted with the encryption
algorithm stored in the Java key store %TAFJ_HOME%\conf\cache\cachestore.keystore file
with the key temn.tafj.cache.ignite.encryption.algo. The value of this key is the encryption

Page 14
TAFJ C a c h e

algorithm that will be used for encrypting data in the caches. Key values within the caches
are not encrypted.

Ma n i p u l a t i n g th e cac h e s t o r e . k e y s t o r e file
If you would like to change the encryption algo, Ignite password, or anything else contained
within the keystore, use the command tKeyStore in %TAFJ_HOME%\bin. Examples follow.

To add or change the Ignite user and password. (If the keystore doesn’t exist it will be
created.)

tKeyStore -keystoretype JCEKS -file C:\data\cachestore.keystore -keystorepassword

mypassword -key Ignite -value IgnitePassword

To add or change the encryption algo (valid values are AES, DES, DES3, RC2, and
BLOWFISH)

tKeyStore -keystoretype JCEKS -file C:\data\cachestore.keystore -keystorepassword

mypassword -key temn.tafj.cache.ignite.encryption.algo -value AES

To encrypt a keystore or Ignite password to use in caching.properties. The value will

be encrypted and then converted to a Base64 value.

tKeyStore –algo AES –initializer myinitializer12345 –value IgnitePassword

will produce:

Encrypted value of IgnitePassword is P9RdygoUjPryZBo/urbd8A==

Page 15
TAFJ C a c h e

Oracl e Webl o g i c Coh e r e n c e Set u p

A gar file is a file that is deployed to Weblogic for Coherence configuration.

In the TAFJCache.gar file found in %TAFJ_HOME%\appserver\weblogic, one must setup the

proxy for pure java clients (non application server clients). Change localhost to the IP
address of the server.

<!-- One or more proxy schemes defined for client connections.

Change address for each server.
See
https://docs.oracle.com/cd/E18686_01/coh.37/e18678/gs_configextend.htm#COHC
G5021 for more info
-->
<proxy-scheme>
<service-name>ExtendTcpProxyService</service-name>
<acceptor-config>
<tcp-acceptor>
<local-address>
<address>localhost</address>
<port>9099</port>
</local-address>
</tcp-acceptor>
</acceptor-config>
<proxy-config>
<cache-service-proxy>
<enabled>true</enabled>
<lock-enabled>true</lock-enabled>
</cache-service-proxy>
<invocation-service-proxy>
<enabled>true</enabled>
</invocation-service-proxy>
</proxy-config>
<autostart>true</autostart>
</proxy-scheme>

Page 16
TAFJ C a c h e

Oracl e Coh e r e n c e Set u p for Clie n t s

Sta n d a l o n e Clie n t s
For clients running pure java client programs such as tRun with Coherence clusters inside or
outside of Weblogic, you must uncomment the following line in tafj_conf.bat (sh) to set up the
JVM properties.

rem # If using oracle coherence

set "JAVA_OPTS=%JAVA_OPTS% -Dcoherence.cacheconfig=tafj-cache-config-client.xml

-Dcoherence.jcache.configuration.classname=passthrough -Dcoherence.log=log4j
-Dcoherence.log.level=1"

Then you must set the following %TAFJ_HOME%\conf\cache\tafj-cache-config-client.xml.

Change localhost to the IP address of the server.

<!-- remote-scheme for Coherence client connections (Weblogic

Coherence must be running). Change address from localhost to IP of server.
See
https://docs.oracle.com/cd/E18686_01/coh.37/e18678/gs_configextend.htm#COHC
G5021 for more info
-->
<remote-cache-scheme>
<scheme-name>extend-remote</scheme-name>
<service-name>ExtendTcpProxyService</service-name>
<initiator-config>
<tcp-initiator>
<remote-addresses>
<socket-address>
<address>localhost</address>
<port>9099</port>
</socket-address>
</remote-addresses>
</tcp-initiator>
<outgoing-message-handler>
<request-timeout>5s</request-timeout>
</outgoing-message-handler>
</initiator-config>
</remote-cache-scheme>

All Clie n t s
The value in CACHENAME for the particular table, matches to a defined cache setup in
Oracle Coherence. For standalone mode (not in a Weblogic application server), this file is
%TAFJ_HOME%\conf\cache\tafj-cache-config-client.xml. It must be noted that cache names
must match what is in the TAFJCache.gar file if targeting Coherence within Weblogic.

As you see with the Tables cache used above in the CACHENAME column, this matches the
definition found in tafj-cache-config.xml. These definitions must match or the cache will not
be found.

<?xml version="1.0"?>

Page 17
TAFJ C a c h e

<!-- This cache configuration file is for the storage enabled and disabled
members in the Managed Coherence Server example. -->
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-
cache-config coherence-cache-config.xsd">

<defaults>

<scope-name>TAFJCache</scope-name>
</defaults>

<caching-scheme-mapping>
<cache-mapping>
<cache-name>Tables</cache-name>
<scheme-name> extend-remote </scheme-name>
</cache-mapping>
</caching-scheme-mapping>

The <scheme-name> will map to an Oracle cache configuration within the same tafj-cache-
config-client.xml. In this case extend-remote refers to a caching scheme on the Coherence
server and was shown earlier in the <remote-cache-scheme> tag.

Once the table is matched within TAFJ (comparing the <cache-name> element against what
is in CACHENAME column in the TAFJ_CACHE table), caching on that table will begin
across a cluster. For Temenos use, “Tables” cacheName should be purely in memory.
Writes on cached data delete the element from the cache, and keys must be re-read. If a
node in the cluster is shutdown, it’s elements will also be removed from the cache (if the data
is not using a replicated scheme), and the data will have to be re-read by other nodes. Using
replicated data does incur a performance cost especially with writing to caches, so be sure
that is the scheme you want to use.

This document does not cover Coherence cache setup (link below). One must be very
careful not to exceed memory assigned and must setup the appropriate expiry times for data
to be refreshed. Again, when data is being written within T24 for a particular record and that
record is cached, the cache is emptied of that key so subsequent reads will get fresh data.

https://docs.oracle.com/middleware/1221/coherence/develop-applications/GUID-2EF76F2B-
ABF2-43AC-BF47-A600564465A2.htm#COHDG5055

Page 18
TAFJ C a c h e

Log g i n g Coh e r e n c e
In standalone mode, %TAFJ_HOME%\bin\tafj_conf.bat (sh) the following properties are set
so coherence logs to the COHERENCE appender in TAFJTrace.properties. If you want
maximum logging, set coherence.log.level=9

-Dcoherence.log=log4j -Dcoherence.log.level=1

Appender in TAFJTrace.properties:

log4j.logger.Coherence=INFO, coherence

De pl o yi n g the TAFJ.gar file

The TAFJCache.gar file can be deployed as a library and referenced within the
TAFJJEE_EAR.ear file, but we’ve seen reloading issues with this approach. Preferably,
deploy TAFJCache.gar in the TAFJJEE_EAR.ear file. To do this, add the following to
weblogic-application.xml:

<module>

<name>TAFJCache</name>

<type>GAR</type>

<path>TAFJCache.gar</path>

</module>

Then put TAFJCache.gar at the root level of TAFJJEE_EAR.ear file.

Run n i n g in Sta n d a l o n e Mod e with Coh e r e n c e

A couple of rules running in Standalone Mode.

1) One should not expect to write data to the database and expect the application server
to know about it if that data is cached at the application server. Hence, it is very
dangerous to write data outside of an application server context where data is
cached.

2) When running Coherence or another provider outside of an application server, it

doesn’t mean you are speaking to the cluster caches that the application server is
using. Most likely, they are different.

In order to run in standalone mode, one must have at least one coherence server running
(you can have multiple if desired). COHERENCE_HOME is the install directory of
Coherence, for example:

C:\ >java -Dcoherence.cacheconfig=%TAFJ_HOME%\conf\cache\tafj-cache-config.xml -cp

%TAFJ_HOME%\conf\cache;%COHERENCE_HOME%\lib\coherence.jar;

Page 19
TAFJ C a c h e

%COHERENCE_HOME%\lib\cache-api.jar;%COHERENCE_HOME%\lib\coherence-
jcache.jar com.tangosol.net.DefaultCacheServer

Note that tafj-cache-config.xml must be found in the classpath. tangosol-coherence-

override-dev.xml in %TAFJ_HOME%\conf\cache overrides the cluster name and sets up
logging to use the TAFJTrace.properties file, but you don’t have to do that.

Now uncomment the following like in tafj_conf.bat or tafj_conf.sh and run tRun or whatever
program from %TAFJ_HOME%\bin.

set "JAVA_OPTS=%JAVA_OPTS% -Dcoherence.cacheconfig=tafj-cache-config.xml

-Dcoherence.jcache.configuration.classname=passthrough"

Page 20
TAFJ C a c h e

Mon i t o r i n g Coh e r e n c e
See the following instructional video by Oracle how to monitor Coherence caches through
jVisualVM.

https://www.youtube.com/watch?v=0BvkFFlZ000

JBos s Dat a Grid Set u p

For EAP version 6 and up, one first has to download jboss-datagrid-6.6.0-eap-modules-
library.zip (or above) from Red Hat and unzip it on top of %JBOSS_HOME%. These
modules are put into a separate slot (ie, jdg-6.6) and must be referenced by the tafj module
definition file.

Add the following modules to %JBOSS_HOME

%\modules\com\temenos\tafj\main\module.xml under the dependencies section after
unzipping jboss-datagrid-6.6.0-eap-modules-library.zip to the %JBOSS_HOME%\modules
directory.

<dependencies>

Etc.

<module name="javax.cache.api" slot="jdg-6.6" services="export"/>

<module name="org.infinispan" slot="jdg-6.6" services="export"/>

<module name="org.infinispan.commons" slot="jdg-6.6" services="export"/>

<module name="org.infinispan.lucene" slot="jdg-6.6" services="export"/>

<module name="org.infinispan.client.hotrod" slot="jdg-6.6" services="export"/>

<module name="org.jgroups" slot="jdg-6.6" services="export"/>

Now download jboss-datagrid-6.6.0-server.zip and unzip it somewhere. This is your Jboss

Data Grid Server and then becomes %JDG_HOME%. Make a back up of either
standalone.xml or clustered.xml in %JDG_HOME%\standalone\configuration. Copy either
%TAFJ_HOME%\appserver\ jboss\jboss6eap\ standalone-JDG.xml or %TAFJ_HOME
%\appserver\ jboss\jboss6eap\clustered-JDG.xml (clustered must be used if Memory Grid
Lock Manager to be used) to %JDG_HOME%\standalone\configuration and reference this
file when starting up Jboss Data Grid Server, ie. .\standalone.bat –c clustered.xml etc.

Example below from standalone-JDG.xml of the caches added for T24 in this file from the
original configuration:

Page 21
TAFJ C a c h e

<subsystem xmlns="urn:infinispan:server:core:6.4">

<cache-container name="local" default-cache="default" statistics="true">

<local-cache name="default" start="EAGER">

<locking striping="false" acquire-timeout="30000" concurrency-level="1000"/>

<transaction mode="NONE"/>

</local-cache>

<local-cache name="Tables" start="EAGER" batching="false">

<locking striping="false" acquire-timeout="20000" concurrency-level="500"/>

<transaction mode="NONE"/>

</local-cache>

</cache-container>

Communication from the EAP appserver to the Jboss Data Grid server occurs through a
HotRod client. Your configuration for Jboss Data Grid Server should depend on what your
EAP configuration is. For example if you have a clustered EAP with two nodes and each
node exists on a separate machine, you might want a clustered Jboss Data Grid Server set
up to match the EAP configuration and use replicated caches. Refer to the Jboss Data Grid
manual for more information.

Note that you must start your jBoss Data Grid on a port offset to avoid conflicts, ie.

.\standalone.bat -c clustered.xml -Djboss.socket.binding.port-offset=100

Note the port HodRod starts on as it must match the value in your caching.properties:

11:55:04,856 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) JDGS010000:

HotRodServer starting

11:55:04,859 INFO [org.infinispan.server.endpoint] (MSC service thread 1-7) JDGS010001:

HotRodServer listening on 127.0.0.1:11322

In caching.properties

………………………………..

temn.tafj.cache.jboss.jdg.port = 11322

Page 22
TAFJ C a c h e

Usi n g with a Sta n d a l o n e clie n t

Download jboss-datagrid-6.6.0-remote-java-client.zip and install it under $TAFJ_HOME/ext.

This is required for DBTools if using Memory Grid Lock Manager for viewing and removing
locks. Start your JBoss DataGrid server. Now running programs will cache data when you
read it in standalone mode.

Page 23
TAFJ C a c h e

Web s p h e r e Extre m e Scal e Grid Set u p

Install latest version of eXtreme Scale Client like the below (as of this document date latest
Fix Pack is 8.6.0.8 which should be used). See Websphere docs for instructions on how to
install, etc.

Start with a new or existing WAS profile. Then augment the profile for ExtremeScale. Do
this with the Websphere Customization Toolbox which under the profile’s start menu “First
Steps.”

Page 24
TAFJ C a c h e

Install Extreme Scale somewhere. This becomes %ES_HOME%. The code within WAS will
act as a client to this server.

Page 25
TAFJ C a c h e

Mod ifyi n g th e Extr e m e S c a l e exa m p l e to add so m e TAFJ

Cac h e s
To allow caches to be allowed to be configured, differently, we put each in its own object
grid. In %ES_HOME%\ObjectGrid\gettingstarted\server\config\objectgrid.xml, modify it to be
the following:

<objectGridConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://ibm.com/ws/objectgrid/config ../objectGrid.xsd"

xmlns="http://ibm.com/ws/objectgrid/config">

<objectGridConfig xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://ibm.com/ws/objectgrid/config ../objectGrid.xsd"

xmlns="http://ibm.com/ws/objectgrid/config">

<objectGrids>

<objectGrid name="MemoryLockingCacheGrid" txTimeout="2400" securityEnabled="false"

txIsolation="READ_COMMITTED">

<backingMap name="MemoryLockingCache" copyMode="COPY_TO_BYTES"

lockStrategy="PESSIMISTIC" lockTimeout="2400000"
numberOfLockBuckets="100"

nullValuesSupported="false" />

</objectGrid>

<objectGrid name="TablesGrid" txTimeout="2400" securityEnabled="false"

txIsolation="READ_COMMITTED">

<backingMap name="Tables" copyMode="COPY_TO_BYTES"

lockStrategy="PESSIMISTIC"

nullValuesSupported="false" />

</objectGrid>

</objectGrids>

Page 26
TAFJ C a c h e

</objectGridConfig>

In %ES_HOME%\ObjectGrid\gettingstarted\server\config\deployment.xml, modify it to be the

following:

<deploymentPolicy xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://ibm.com/ws/objectgrid/deploymentPolicy
../deploymentPolicy.xsd"

xmlns="http://ibm.com/ws/objectgrid/deploymentPolicy">

<objectgridDeployment objectgridName="MemoryLockingCacheGrid">

<mapSet name="mapSet" numberOfPartitions="13" minSyncReplicas="0"

maxSyncReplicas="1" >

<map ref="MemoryLockingCache"/>

</mapSet>

</objectgridDeployment>

<objectgridDeployment objectgridName="TablesGrid">

<mapSet name="mapSet" numberOfPartitions="13" minSyncReplicas="0"

maxSyncReplicas="1" >

<map ref="Tables"/>

</mapSet>

</objectgridDeployment>

</deploymentPolicy>

Star ti n g your Catal o g serv e r

In %ES_HOME%\ObjectGrid\gettingstarted, run startcat.bat. Notice the port and domain
which will be important when connecting from Websphere.

[8/5/16 14:44:12:840 CEST] 00000001 XIORegistry I CWOBJ9054I: The eXtremeIO

registry is using the endpoint ID [47d141565abb8099e000605718b043a2].

[8/5/16 14:44:13:747 CEST] 00000001 TCPChannel A TCPC0001I: TCP Channel

XIOInboundTCP-47d141565abb8099e000605718b043a2 is listening on host * (I

Pv6) port 2809.

[8/5/16 14:44:13:750 CEST] 00000001 ServerImpl I CWOBJ2518I: Starting the

ObjectGrid catalog service: cs0 for domain DefaultDomain.

Page 27
TAFJ C a c h e

[8/5/16 14:44:13:750 CEST] 00000001 ServerImpl I CWOBJ1252I: Quorum is disabled for

the catalog service.

[8/5/16 14:44:15:004 CEST] 00000001 JvmMemoryUtil I CWOBJ4542I: Basic BackingMap

memory size was enabled.

[8/5/16 14:44:15:036 CEST] 00000001 ServerImpl I CWOBJ3001I: The ObjectGrid

EntityManager service is available to process requests for ObjectGrid

: BalanceGrid and container or server: cs0

[8/5/16 14:44:15:046 CEST] 00000001 CatalogServic I CWOBJ8263I: The management

concentrator MBean is enabled for the cs0 catalog server.

[8/5/16 14:44:15:182 CEST] 00000001 CatalogServic I CWOBJ8106I: The master catalog

service cluster activated with cluster DefaultDomain

Star ti n g a Cont a i n e r
In %ES_HOME%\ObjectGrid\gettingstarted, run “startcontainer.bat TemenosGrid”. Then run
“startcontainer.bat MemoryLockingCacheGrid.”

Now you should have a catalog and two containers running, let’s now connect to it through
Websphere.

Se t u p a Catal o g Servi c e Do m a i n in WAS

This will point to your ObjectGrid server. When you started your catalog server in the
previous step

It should look like the below when finished. Check that the connection works. If it doesn’t
check the ports as previously explained.

Page 28
TAFJ C a c h e

If you have set your tafj.properties correctly as explained in “Property Setup” you should see
information going to %TAFJ_HOME%\log\cache.log (if loglevel is set to DEBUG). Example:

[INFO ] 2016-08-05 15:10:37,576 [SIBJMSRAThreadPool : 1] CACHE - Getting CatalogDomainInfo for domain:

DefaultDomain

[INFO ] 2016-08-05 15:10:37,671 [SIBJMSRAThreadPool : 1] CACHE - Connecting with domain manager in app
server mode for domain: DefaultDomain

[DEBUG] 2016-08-05 15:10:37,813 [SIBJMSRAThreadPool : 1] CACHE - Returning Websphere cache: Tables

[INFO ] 2016-08-05 15:10:37,850 [SIBJMSRAThreadPool : 1] CACHE - Read : F.COMPANY.CHECK:MASTER

from cache: Tables

[DEBUG] 2016-08-05 15:10:37,872 [SIBJMSRAThreadPool : 1] CACHE - Reading :

F.TEC.ITEMS:LOCK.COLLISION[1825380092]

[INFO ] 2016-08-05 15:10:37,898 [SIBJMSRAThreadPool : 1] CACHE - Read : F.TEC.ITEMS:LOCK.COLLISION

from cache: Tables

Sta n d a l o n e Extre m e Scal e

Probably the easiest way to get up and running to see TAFJ working with Extreme Scale is to
download it somewhere, unzip it, which then becomes %ES_HOME% as mentioned
previously.

If you’ve done the above steps, and if your properties and tables are set up correctly, you
should be able to run T24 programs and see Caching happening with the cache.log file in
%TAFJ_HOME%\log if the CACHE logger is set to DEBUG.

Usi n g TAFJCac h e as a Com p o n e n t

As mentioned previously, TAFJCache.jar can be used as an independent component. It’s
only dependencies are TAFJLogging.jar, log4j-api.jar, and log4j-core.jar, plus any
appropriate third party client libraries. This will enable you to write code such as:

Page 29
TAFJ C a c h e

import com.temenos.tafj.cache.CacheConstants;

import com.temenos.tafj.cache.CacheParameters;

import com.temenos.tafj.cache.ITAFJCache;

import com.temenos.tafj.cache.TAFJCacheFactory;

public class TestCache {

public static void main(String [] args) {

CacheParameters params = new CacheParameters("MYCACHE");

//params.setCacheProvider(CacheConstants.CACHE_PROVIDER_TEMENOS_LOCAL);

//params.setCacheProvider(CacheConstants.CACHE_PROVIDER_JBOSS);

//params.setCacheProvider(CacheConstants.CACHE_PROVIDER_COHERENCE);

params.setCacheProvider(CacheConstants.CACHE_PROVIDER_WAS);

// Alternatively...

//params.setCacheProvider(TAFJCacheFactory.getCacheProperties().getProperty

// (CacheConstants.KEY_PROP_CACHE_EXTERNAL_CACHING_PROVIDER));

ITAFJCache cache = TAFJCacheFactory.getCache(params);

cache.put("Key1", "Value1");

String value = (String)cache.get("Key1");

System.out.println(value);

Above, a CacheParameters object is created and a provider is chosen (in this case WAS, so
extremeScale is the target cache server so objectgrid.jar would be a dependency).
Alternatively, the provider could have been retrieved from the caching.properties file.
Properties set on the CacheParameters object will be overridden by any properties unless
setOverrideParams(true) is called. Once a cache is retrieved by the TAFJCacheFactory, use
it to cache your objects.

Page 30
TAFJ C a c h e

Mon i t o r i n g Cac h i n g
One can monitor cache time in the TAFJ monitor by comparing cache read time against
regular read time and number of cache reads against number of regular reads as shown
below.

App e n d i x
Esti m a t i n g tabl e siz e s
For Oracle use the query :

select table_name, round(blocks*8192/(1024*1024)) SIZE_MB from user_tables where

table_name = 'F_AC_BALANCE_TYPE'

TABLE_NAME SIZE_MB

----------------- ----------

F_AC_BALANCE_TYPE 2

This gives you the size in megabytes of F_AC_BALANCE_TYPE. So the approximate

amount of memory if the whole table is cached is 2 MB. Add enough memory to your
application server or data grid server so you don’t have a problem.

Page 31