JBossDeveloper

I'm confused about what pushStateWhenCoordinator does -- doesn't it mean taking all in-memory state and doing a write-through to the cache loader?

_getState doesn't provide that kind of behavior.

No, this won't work. The byte[] returned by _getState() is made up of subsidiary byte[]s for the in-memory state and the persistent state. The _setState method is responsible for parsing these out and doing the appropriate stuff with each, including passing the *persistent* state to the cache loader. But CacheLoader.storeEntireState() only knows how to handle the persistent state.

Actually, as I wrote this I realized that for 1.2.4 I did work to make the data format of the in-memory transfer and the persistent transfer consistent. So *maybe* it will work -- you parse out the the in-memory state and pass it to the cache loader.

This is bad though -- the fact that the data structure of the persistent state and the in-memory state is the same is 1) not part of any interface contract, and therefore 2) if true for the cache loaders we ship with, it's only because *we* wrote them that way. Other custom implementations may be different.

Re: the timeout and force parameters. I've no opinion on whether the timeout should be configurable, or whether you should just use StateTransferTimeout. The combination of "timeout" and "force" was meant to allow the calling code to loop with a successively longer timeout; on the final iteration of the loop, "force" would be set to true, and at that point _getState would force the breakage of any locks in order to ensure the transfer goes through. Perhaps you should follow a similar approach.

Well, one thought is that going to 2.0 gives us the freedom to specify in the CacheLoader contract what the format of the data passed to storeEntireState has to be. So, we theoretically could take the in-memory state and pass it to storeEntireState. But we'd all have to think very long and very hard about doing that, as we'd have to live by the contract as much as anyone.

Hmm, that would involve rewriting the current set of cache loaders. I'm concerned about feasibility wrt. timescales here.

Well, one thought is that going to 2.0 ...

What about something like this -

1) New node becomes coord
2) Triggers a process in the singleton cache loader to take over ownership of the singleton cache.
3) Starts by walking the tree to get a picture of the in-memory state (by doing _get? Bad, since no locks are obtained. Perhaps by passing in a flag to bypass cache loading?)
4) Make calls to cacheLoader.put() for each node visited in the tree.

Yes, this is brute-force and slower than getting/setting state as bytes, but at least this sticks with data format contracts without having to make assumptions.

Thoughts?

I think what you propose is the way to go and may not be particularly slower than using the byte arrays. In the end a state transfer of in-memory state involves walking the tree on the side providing the state, and storing persistent state involves executing the cache loader's code for storing each node. The byte arrays just package up the data for efficient transmission over the wire.