How do I use Infinispan as a data grid?

[cdman]

Hello.

I'm trying to use Infinispan as an in-memory data grid and my current plan is the following: start a set of HotRod servers in DIST mode and use custom interceptors (2) to create / modify objects when command objects are written to the server. My questions would be:

• Will this work? Is it possible to interact with the cache (ie. to modify other objects) from an interceptor?
• Which interceptor should I use? handlePutMapCommand and handleReplaceCommand from the PrePostProcessingCommandInterceptor (3) look promising
• Is this the best design? I'm new to Infinispan so I might have missed some options
• Is is possible to start a HotRod server programatically (ie from your code rather than from the shell script?) This would make debugging easier for me (and also conceptually I'm thinking of the application as a separate entity, not something hosted inside of a container)

PS. This is part of a bigger question (1) which I've broken up so that it is easier to "digest". Thank you in advance for your help.

(1) http://community.jboss.org/thread/167245

(2) http://community.jboss.org/docs/DOC-14852

(3) http://docs.jboss.org/infinispan/4.0/apidocs/org/infinispan/interceptors/base/PrePostProcessingCommandInterceptor.html#handlePutMapCommand%28org.infinispan.context.InvocationContext,%20org.infinispan.commands.write.PutMapCommand%29

[mircea.markus]

Will this work? Is it possible to interact with the cache (ie. to modify other objects) from an interceptor?

Should be, but the prefared way is through the Cache interface.

Which interceptor should I use? handlePutMapCommand and handleReplaceCommand from the PrePostProcessingCommandInterceptor (3) look promising

Look at an existing interceptor implementation, e.g. CacheMgmtInterceptor:http://bit.ly/mHQPfx.

Generally "handlePutMapCommand" is invoked on a cache.putAll(Map).

Is is possible to start a HotRod server programatically (ie from your code rather than from the shell script?) This would make debugging easier for me (and also conceptually I'm thinking of the application as a separate entity, not something hosted inside of a container)

Yes, take a look at a test in the HotrodClient test suite, e.g.  http://bit.ly/kFBoqL

[cdman]

Hello,

Thank you for the feedback. It has been a long time coming, but finally I put together a concrete code example using some of techniques from above. You can read the article about it here: http://www.todaysoftmag.com/article/en/12/High_availability_performance_systems_using_data_grids_in_Java_437 and get the code here: https://github.com/cdman/infinispan-exchange

I would also like to thank Dan Berindei for his help.

Some updates:

• hotrod is not used because watching for elements would have needed implementing an interceptor (I understand from Dan that listeners are called before the element is actually committed and using them would have created the risk of double-processing). Instead I just created a REST endpoint
• also, hotrod doesn't support continious query, so there is no good solution for the output
• an other issue with HotRod (in my opinion) is that implementation of non-java clients seems to have stalled a little bit
• the transaction + failover support seems to have some bugs which Dan is (has?) ironed out

Unfortunately the performance I got out of the system (~1000 TPS) is too low for the kind of solution I was looking for (feel free to look at the code and suggest improvements, but the matching engine itself supports ~400k operations on the same system and a test with Radargun confirmed that this value is in the range of values to be expected).

Thank you,

Attila

[mircea.markus]

thanks for sharing Attila. I'll take a look.