Hibernate/Infinispan Cache Transactions and Cluster question

[bjve]

Two questions on the Infinispan 2nd level Hibernate cache provider:

* I'm not using a J2EE server, but it looks like the Infinispan 2nd level cache provider requires a TransactionManager/TransactionManagerLookup to be set up. Is it possible to use the Infinispan 2nd level cache outside of a J2EE server, and if so how do I set up the transaction manager lookup?

* I'm adding the Infinispan 2nd level cache provider to existing servers that already use JGroups. Should I set Infinispan to use the same JGroups cluster, or should I use two separate cluster names?

Thanks.

[galder.zamarreno]

"bjve" wrote:
Two questions on the Infinispan 2nd level Hibernate cache provider:

* I'm not using a J2EE server, but it looks like the Infinispan 2nd level cache provider requires a TransactionManager/TransactionManagerLookup to be set up. Is it possible to use the Infinispan 2nd level cache outside of a J2EE server, and if so how do I set up the transaction manager lookup?

I've just added a FAQ entry to our wiki explaining this, see http://is.gd/58UvB or one of the Hibernate 2nd level cache FAQ entries in http://www.jboss.org/community/docs/DOC-13439.

As explained there, an example like this, where a production level JTA transaction manager is used (i.e. JBoss Transaction) is currently not available but based on the guidance provided, it should be easy to build one up.

* I'm adding the Infinispan 2nd level cache provider to existing servers that already use JGroups. Should I set Infinispan to use the same JGroups cluster, or should I use two separate cluster names?

I'll answer this asap.

[bjve]

Hi Galder -- thanks again for the advice. Two followups:

First, my hibernate/infinispan code is running in a standalone server (no J2EE). I am using the default JDBCTransactionFactory that comes with Hibernate and have not set any transaction manager lookup in my config, so the transactionManager returned by HibernateTransactionManagerLookup will be null. Do I need to set up a transaction manager lookup for the Infinispan 2nd level cache to work properly in transactional mode, or will the JDBCTransactionFactory that I'm using be sufficient?

Next, re: collection caching: I noticed that my collection caches weren't being created by the InfinispanRegionFactory (and no exceptions were thrown when I set custom eviction settings for them), so I looked through SessionFactoryImpl in the Hibernate code and saw that I have to explicitly set <cache usage=transactional/> on each collection, even if the owning entity has the transactional cache usage set (otherwise SessionFactoryImpl will never call buildCollectionRegion and collection caching won't get turned on). Is this the intended behavior (that the cache usage has to be set explicitly on each collection)? If so it might be useful to throw an exception in the InfinispanRegionFactory if the cache usage isn't set up as above, since in my case I had custom eviction settings for a collection that referred to a cache region that didn't exist.

Thanks,
Brian

[bjve]

Hi Galder -- also a question about sharing the same JGroups cluster. As you suggested I set my own cluster and the Infinispan cluster to use the same cluster name, but this is resulting in a lot of exceptions being thrown in CommandAwareRpcDispatcher.handle(), where the Infinispan code tries to deserialize my own non-Infinispan messages and fails. This seems like it might be wasting a lot of cycles (and generating a ton of exceptions). Please let me know if you have any thoughts on the performance of sharing the same cluster vs. just separating into two separate cluster names, in which case JGroups would be able to more easily ignore messages intended for the other cluster. Thanks,
Brian

[galder.zamarreno]

"bjve" wrote:
First, my hibernate/infinispan code is running in a standalone server (no J2EE). I am using the default JDBCTransactionFactory that comes with Hibernate and have not set any transaction manager lookup in my config, so the transactionManager returned by HibernateTransactionManagerLookup will be null. Do I need to set up a transaction manager lookup for the Infinispan 2nd level cache to work properly in transactional mode, or will the JDBCTransactionFactory that I'm using be sufficient?

I have just updated the FAQ entry in http://www.jboss.org/community/docs/DOC-13439 cos I had confused myself with transaction factory and transaction manager lookup. Setting JDBCTransactionFactory is not sufficient since it's not a set up any transaction manager, it is not aware of XA resources or similar.

To be more precise, you could use JDBCTransactionFactory and configure Infinispan with a transaction manager but any combined transactional work would just be isolated, since Hibernate would be using a transaction and Infinispan a different one, so each would work independently from an transactional perspective.

Next, re: collection caching: I noticed that my collection caches weren't being created by the InfinispanRegionFactory (and no exceptions were thrown when I set custom eviction settings for them), so I looked through SessionFactoryImpl in the Hibernate code and saw that I have to explicitly set <cache usage=transactional/> on each collection, even if the owning entity has the transactional cache usage set (otherwise SessionFactoryImpl will never call buildCollectionRegion and collection caching won't get turned on). Is this the intended behavior (that the cache usage has to be set explicitly on each collection)? If so it might be useful to throw an exception in the InfinispanRegionFactory if the cache usage isn't set up as above, since in my case I had custom eviction settings for a collection that referred to a cache region that didn't exist.

Hmmm, I suppose InfinispanRegionFactory might be able to check for any custom settings for any entity/collection and do a check for it, i.e. check if cache usage is transactional or read-only. Any other time where there's no specific setting per entity/collection, I don't think we can do that.

I'll look into it: http://opensource.atlassian.com/projects/hibernate/browse/HHH-4637

[galder.zamarreno]

"bjve" wrote:
Hi Galder -- also a question about sharing the same JGroups cluster. As you suggested I set my own cluster and the Infinispan cluster to use the same cluster name, but this is resulting in a lot of exceptions being thrown in CommandAwareRpcDispatcher.handle(), where the Infinispan code tries to deserialize my own non-Infinispan messages and fails. This seems like it might be wasting a lot of cycles (and generating a ton of exceptions). Please let me know if you have any thoughts on the performance of sharing the same cluster vs. just separating into two separate cluster names, in which case JGroups would be able to more easily ignore messages intended for the other cluster. Thanks,
Brian

Brian, the thing is that you shouldn't use the same cluster name. Instead, there should be a way to define the shared transport name or multiplexer channel. This is what I'm discussing in http://lists.jboss.org/pipermail/infinispan-dev/2009-December/001996.html because it's less than clear how to do that in Infinispan right now.

In the mean time, use different JGroups configurations each with different multicast address, port...etc. You can specify this either via configurationFile property, i.e.

<global>
 <transport transportClass = "org.infinispan.remoting.transport.jgroups.JGroupsTransport"
 clusterName="infinispan-hibernate-cluster" distributedSyncTimeout="50000">
 <!-- Note that the JGroups transport uses sensible defaults if no configuration property is defined. -->
 <properties>
 <!-- TODO: Change to udp.xml once streaming transfer requirement has been removed. -->
 <property name="configurationFile" value="flush-udp.xml"/>
 </properties>
 <!-- See the JGroupsTransport javadocs for more flags -->
 </transport>
 </global>

Or via configurationXml where you put the JGroups XML as String, in the value part of the property.

[bjve]

Thanks -- so just to confirm on the transaction manager lookup question: if I am OK with the cache invalidation being done in a separate transaction from the database transaction, then I do not need to set up a transaction manager lookup? So in that case I would just set up Hibernate to use the JDBCTransactionFactory with no transaction manager lookup.

Again, it is fine for this project if the cache invalidation is not in the same transaction as the database transaction -- I just want to confirm that the Infinispan 2nd level cache provider will still be able to invalidate entries in the other caches in the cluster properly even if there is no transaction manager lookup.

[galder.zamarreno]

The problem with synchronous invalidation or replications if not using a transaction manager in infinispan is that if one of the nodes fails to apply the changes, this won't cause the others to rollback. When an infinispan TM has been configured, this is done in a 2PC way, so even if one fails, the others get rollbacked.

Now, in your case, if you want your invalidation to be transactional, you do need to configure Infinispan with a transaction manager lookup class because in this case it's not receiving it from Hibernate. So, to sum up, you'd configure:

- Transaction manager lookup for infininispan via infinispan's configuration file.
- Transaction factory for Hibernate cfg.
- No transaction manager lookup for Hibernate

The transaction that Infinispan uses and the one Hibernate uses would be different.

[pb00067]

Hi Brian,

same as you I'm trying to use Hibernate/Infinispan outside any application server.

From my experience it is very dangerous to use Infinispan as 2Lcache implementation for Hibernate

without using JTA integration.

if the cache invalidation is not in the same transaction as the database transaction,

you easily will run into having the 2LCache in a inconsistent state!

See following flush - rollback scenario :

session1: create and persist a            // a being an instance of a cached persistent class

session1: a.setName("FirstValue");

session1: commit

session2: read a

session2: a.setName("Modified");

session2: flush  // note that flush also may be called implicitely by do queries, here the objects are written into 2L Cache

session2: rollback

   // Note that without JTA integration, without specifying transaction manager lookup, here Infinispan don't get noticed about the rollback

session3: read a

session3: a.name == "Modified"    ---> WRONG!!

regards

Guenther Demetz