Eviction and FetchStateOnStartup

[ramvaidya]

In an installation of a JGroup of TreeCaches, if one Cache (in my case a JBoss MBean) evicts a node and others don't the Cache operation is documented and understood.
Now if yet another TreeCache instance joins this group with FetchStateOnStartup set to true, the state it grabs is non-deterministic because it depends on cordinator's eviction state and then a get on evicted node doesn't cause replication from other nodes.
Is this expected or is it un unitended feature?

[belaban]

This is expected behavior. This case requires a cache loader which can on-demand reload elements that have been evicted.
I understand you can configure eviction policies to evict across the cluster, Ben ?

Bela

[ben.wang]

No, currently we don't have the option of evicting across the cluster. This is because different node caches may have different get operation which is local. So if we evict across the group, you have an element on Cache A that is just being touched (i.g., get) while it is being evicted from Cache B since it has not been touched for a while. This can cause surprised behavior as well.

If there is a good solution, maybe I will implement in 1.1 release.

-Ben