Batching multiple updates on the same key

[infinikli]

Hi!

1. We have an environment where we're frequently putting data into cache with the same key. This is because we use an event-driven application and we are forced to listen to those events. Let's say we fire Event A ten times in ten seconds. This would lead into ten RPC-calls on one key. Our data has no real-time-requirements and it is fine with us to update the data delayed. I've seen the ReplicationQueue but I'm not sure whether it fulfills our requirements: the ReplicationQueue would buffer the ten UpdateCommands and then do a flush after the replQueueInterval has been hit and then perform ten UpdateCommands. What we need is not a queue but some kind of a java.util.Set-implementation which allows only one entry per key and command-type.

2. We don't want our business-application to do caching, because we need all heap for this application. So we set up a client-server-modell using Hotrod. But actually I do not understand the difference between a Hotrod-client-configuration and a normal cache-configuration with having evictionMaxEntries=0.In addition, when using a HotrodClient all the features requested in 1. would not make many sense because they only affect Infinispan-architecture (right?).

3. In a nutshell: should I consider implementing an own buffer on business-application-level (primarily when using Hotrod)?

Many thanks!

[sannegrinovero]

It seems you want something similar to what we implemented for ISPN-619, which was basically implementing this concept for CacheLoaders (as they are assumed to be much slower than Infinispan).

Currently commands are set right away, and the ReplicationQueue works at the messag layer.

The short answer is yes, you should implement your own buffer.

Still I like your idea, and it sounds something reasonable to implement on HotRod.. could you please open a feature request on JIRA?

If you would you like to try implementing it and contribute it you would benefit from our directions, reviews and some other hundreds of testers

This might make sense not only on HotRod level, bet even when any comunication happens across nodes:

https://issues.jboss.org/browse/ISPN-159

[mircea.markus]

re:1, that is on our radar ISPN-503

Actually the coalesced logic is already present in the code(ISPN-343), it is only a matter of reusing it for repl queue as well.

If you fill like contributing this would be great!