Does Infinispan require distributed tx (XA Resource)?

[martin.ekstrom75]

Hi

I'm trying to utilize Infinispan 4.1 as a 2LC under Hibernate 3.3 control (Hibernate is configured to use JTA transactions) in a clustered Glassfish 2.1.1 enviroment. I can see the cached instances in JMX - Jconsole. The code did also run fine using EHCache config earlier. However, when changed the persistence.xml config to below, I get lots of exceptions like: Local transaction already has 1 non-XA Resource. I didn't get these exceptions using the ehcache config.

<persistence-unit name="entityManager" transaction-type="JTA">
      <provider>org.hibernate.ejb.HibernatePersistence</provider>
      <jta-data-source>jdbc/ejb/eur_unbranded</jta-data-source>

...

<!-- infinispan config -->

<property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.SunONETransactionManagerLookup"/>

<property name="hibernate.cache.use_second_level_cache" value="true"/>

<property name="hibernate.cache.region.factory_class" value="org.hibernate.cache.infinispan.InfinispanRegionFactory"/>
<property name="hibernate.cache.infinispan.statistics" value="true"/>

So my question is: Does Infinispan require XA resource to be defined for the connection pools? Or can I use a regular data resource (non distributed)?

Thanks in advance!

Martin

[mircea.markus]

This might have to do with this: http://forums.java.net/jive/thread.jspa?threadID=17221

[martin.ekstrom75]

But since I use Infinispan as a 2LC under Hibernate, can't Infinispan just hook on to the already existent local transaction? Or does Infinispan always require an XA data source - ie it's not possible to just use a non-XA resource? I mean, the same code works fine using EHCache without XA-resource.

[martin.ekstrom75]

To clearify, the thread http://forums.java.net/jive/thread.jspa?threadID=17221 states that either don't use transactions or make use of distributed transactions in order to have things to work. But, since it worked fine using non distributed tranasctions earlier with ehcache, I wan't to run the system in the same manner now, i.e., to use transactions, but not distributed once. Just setting up a db-pool using a plain javax.sql.DataSource (not javax.sql.XADataSource.

[mircea.markus]

As per the thread, the root of the problem seems to be the fact that you "set the transaction level of my Connector Connection Pool to LocalTransaction and I set SupportsXA to false in the RA properties". If you don't want to do that, the following might work:

- in infinispan configuration, from the transaction tag, remove the transactionManagerLookupClass attribute

- enable invocation batching:

<invocationBatching enabled="true"/>

HTH,

Mircea

[henk53]

Martin Ekström wrote:

To clearify, the thread http://forums.java.net/jive/thread.jspa?threadID=17221 states that either don't use transactions or make use of distributed transactions in order to have things to work. But, since it worked fine using non distributed tranasctions earlier with ehcache, I wan't to run the system in the same manner now, i.e., to use transactions, but not distributed once. Just setting up a db-pool using a plain javax.sql.DataSource (not javax.sql.XADataSource.

Maybe we developers should not fear XADataSource too much. As you can read in the following thread, for a long time we have been indoctrinated somewhat to be afraid of XA, but apparently this is not entirely justified: http://community.jboss.org/thread/151380

I do wonder what the root problem is in this case. Is it the fact that Infinispan enlists itself as a separate participant in the transaction, but Infinispan itself is not an XA resource? In that case you would have two non-XA resources participating in a transaction, which is of course not allowed.

This one suggest that Infinispan is not XA compliant: http://community.jboss.org/thread/146401

If you used CacheConcurrencyStrategy.TRANSACTIONAL with ehcache and that worked in combination with a non-XA datasource within a transaction, then I can think of a couple of reasons:

1. ehcache is a full XA resource itself, therefor the transaction would automatically become an XA transaction anyway and you are fearing something that you actually were already using without knowing it.
2. ehcache (in an older version?) didn't actually supported CacheConcurrencyStrategy.TRANSACTIONAL and silently downgraded this to some other strategy.
3. ehcache simply doesn't register itself as a participant in the transaction, but somehow piggybacks on the datasource for which it represents the cache. I'm not sure if you can call a cache truly transactional if it does a trick like this.

I'm not really an expert here, so take my words with a grain of salt

[jhalliday]

When using transactions in a  db+cache architecture there is a critical distinction between jvm local caches and distributed caches.

In-process caches i.e. the single jvm case, can use a Synchronization for the cache and an XAResource for the database. That's very efficient but relies on the state of the cache being volatile - it will simply vanish in the event of a crash and be recreated on restart by querying the db. Hence the db is the only thing that needs recoverability and thus XA. In the case where there are no other systems in the transaction i.e. no second db or a messaging system, the transaction can actually use a non-XA connection to the db or use XA with one phase commit which is broadly the same. JBoss cache worked this way last time I checked. It's also bacially how hibernate's session level cache works - it registers a Synchronization with the tx so it gets a callback to tell it when to flush to the db before a tx commit. The Synchronization is just an event notification, not an agreement protocol, except in so far as beforeCompletion errors cause a rollback.

Where a single logical cache is spanned (clustered) over multiple JVMs i.e. its state is distributed, then the situation is more complex. A crash may take out only some of the cache nodes and leave others locked and a recovery protocol is needed to ensure things are returned to a consistent state in such case. This is the tricky bit that most cache products are only recently getting to grips with. The cache (and database) has to implement XAResource, not Synchronization. For infinispan I think it's ISPN-272 due in 5.0. Note that a cache can implement XAResource but not recovery, which will make it appear to work fine e.g. locking/versioning, isolation and such, until you get an actual crash. Read the smallprint carefully and test it even more carefully. IIRC ehcache added XAResource in 2.0. I have no idea how good their recovery is, I've never tested it. Without proper XA on the database and distributed cache, you can wind up with the two getting out of sync. This is usually considered a Bad Thing. Hence you should avoid using non-XA databases with a distributed cache.

[galder.zamarreno]

For interest of the audience, as indicated in http://community.jboss.org/docs/DOC-14105 using Infinispan as 2LC for Hibernate without proper transaction management configuration can result in unexpected behaivour.

[henk53]

Jonathan Halliday wrote:

In-process caches i.e. the single jvm case, can use a Synchronization for the cache and an XAResource for the database.

[...]

Where a single logical cache is spanned (clustered) over multiple JVMs [...] the cache (and database) has to implement XAResource, not Synchronization.

I hear you and that sounds very logical. However, in this article about infinispan transactions, only XAResource is being mentioned and there is no reference to Synchronization.

Is the article not complete, or does Infinispan now always uses XAResource when used within a JTA transaction?

[mircea.markus]

Synchronizations are now(5.0) supported: http://community.jboss.org/wiki/InfinispanTransactions#Enlisting_Synchronization