Seeking design advice on using infinispan distributed cache and query

[yangju]

I am looking for advice on our high level design to see if we can leverage infinispan as our distributed cache provider.

We have multiple nodes (4 to 8) in a cluster. On each node there are 10 or 20 threads running to grab business objects that are stored in the cache. The cache is distributed. Therefore each thread can access the business object that stored in any of the node. If the cache is empty, one of the threads will call a complicated jpa query to load N records from databases. It will load records from multiple databases (Shard). So at first look, a cache loader from infinispan does not seem a fit.

There is one issues that makes me wonder if infinispan and its query/indexing are a good fit.

The business objects that are stored in the cache can only be obtained by one thread. First the thread will issue a lucene search query to find a candidate from the cache/index. Once it find one it will try to call cache.remove(key) to remove it from the cache so that other threads (local or from other jvms) won't grab this object.  Once the cache is empty, the thread will try to load another batch of objects from the databases into cache. Therefore, the cache is quite dynamic and constantly changing. Does this cause the query index keeps changing? Does this cache.put(key) causes the index constantly updating? I have found that updating index is slow in our POC.

Any advice or suggestion is greatly appreciated.

We are currently using jboss AS 6 final, infinispan 5.0.0.CR6.

[sannegrinovero]

I am looking for advice on our high level design to see if we can leverage infinispan as our distributed cache provider.

We have multiple nodes (4 to 8) in a cluster. On each node there are 10 or 20 threads running to grab business objects that are stored in the cache. The cache is distributed. Therefore each thread can access the business object that stored in any of the node. If the cache is empty, one of the threads will call a complicated jpa query to load N records from databases. It will load records from multiple databases (Shard). So at first look, a cache loader from infinispan does not seem a fit.

Any advice or suggestion is greatly appreciated.

You could write your own cacheloader, the interface is very simple and the configuration to enable it is pretty easy. Of course loading the data yourself after checking the cache is fine too.

There is one issues that makes me wonder if infinispan and its query/indexing are a good fit.

The business objects that are stored in the cache can only be obtained by one thread. First the thread will issue a lucene search query to find a candidate from the cache/index. Once it find one it will try to call cache.remove(key) to remove it from the cache so that other threads (local or from other jvms) won't grab this object.  Once the cache is empty, the thread will try to load another batch of objects from the databases into cache. Therefore, the cache is quite dynamic and constantly changing. Does this cause the query index keeps changing? Does this cache.put(key) causes the index constantly updating? I have found that updating index is slow in our POC.

Any advice or suggestion is greatly appreciated.

We are currently using jboss AS 6 final, infinispan 5.0.0.CR6.

Yes that means the query index keeps changing, and yes updating the index is an expensive operation, but have you had a look into Hibernate Search's documentation about tuning this process? It can be quite fast, depending on your needs it might be fast enough.

After removing the object from the cache, will you add it back again later? In that case maybe just locking it is a better option.

If you're removing them temporarily and adding in batches, then what do you need the index for? What is the use case?

[pmuir]

Marking this post as assumed answered as no activity from OP. OP, feel free to reopen if still an issue for you