State transfer retrieval and transaction processing

[jarquey]

Hi all,

For our application we need two replicated caches with each node having its own store. We don?t need eviction policy : all the objects are always in memory. The data store avalaible with the cache loader, is used as a persistent storage needed only for ?cold? restarts of the application. I saw that asynchronous cache loader is avalaible in CVS and this feature will be important for me to improve performances.

But a JBossCache feature remains blocking due to the high availability requirements of our application. During state transfer between cluster nodes no new transaction is allowed. So, if a node fails, when it restarts, the application will remain unavailable during all the state transfer (and as we have a big cache it could be during a long time ...) .
Is there a way to override that and allow the continuation of transaction processing on the coordinator node, during node synchronization ?

Thanks

[belaban]

We have to lock the cache during state transfer, otherwise you would get inconsistent state.
If you can transfer the state into the underlying CacheLoader yourself, e.g. though the application, you could turn transferring persistent state off, and then wouldn't have to incur locking of the cache

[jarquey]

In the solution you suggest it will be application responsibility to manage the cache consistency during transfer ? I don?t see how to manage it ?

About cache lock during transfer :
Is it stupid to imagine that the JbossCache coordinator will manage only iterative Treecache nodes locks (for instance at Treecache root level) instead of a global cache lock during synchronization ? Sync time could be longer but the sync could remain secure, allow concurrent transactions and avoid application unavailability.

[belaban]

"jarquey" wrote:
In the solution you suggest it will be application responsibility to manage the cache consistency during transfer ? I don?t see how to manage it ?

The application could for example communicate with all members to stop modifying the cache until the state transfer (e.g. DB copy) is done. However, this boils down to what we already have.
There is no way around locking the entire tree until the state has been copied, at least not IMO.

About cache lock during transfer :
Is it stupid to imagine that the JbossCache coordinator will manage only iterative Treecache nodes locks (for instance at Treecache root level) instead of a global cache lock during synchronization ? Sync time could be longer but the sync could remain secure, allow concurrent transactions and avoid application unavailability.

The problem here is that you can get inconsistencies when you lock node X to get its state, then unlock, but before you transfer the state, someone else modifies X.
I'm not sure it *cannot* be done, but it would have to result on consistent state snapshots.
Note that this problem will somewhat be mitigated with optimistic locking (scheduled for 1.2.5, Oct 2005): we will still need to lock the entire tree, but because there are no other locks in the tree (except when a TX commits), this should be okay.