Risks of Disabling Rehash

[shane_dev]

Can someone confirm that disabling rehash leaves the cluster vulnerable to cache misses?

I've looked at the code and it appears that even the hash wheel is updated, there is no logic for handling misses.

For example, we have an object on node 3. We then add node 4. The object's hash will now map to node 4. However, the object was actually never moved to node 4.

Looking at the code, it appears that the requests are still sent to the nodes found using DistributionManager.locate(Object key) and ultimately ConsistentHash.locate(Object key, int replCount). I don't see anything in the DistributionInterceptor or CommandAwareRpcDispatcher/ReplicationTask that may account for this.

For example, if it is not found on the specified nodes then retry on ALL nodes.

Am I correct that this is an inherent risk or am I simply missing something here.

Thanks,

Shane

[mircea.markus]

For example, we have an object on node 3. We then add node 4. The object's hash will now map to node 4. However, the object was actually never moved to node 4.

If <hash rehashEnabled="false" ../> this is to be expected.  Rehashing does just that: on topology changes, migrate entries to their new owners. Disabling it makes the topology changes more lightweight/quick, but also leaves creates inconsistencies as the one you've described. I would say only disable rehash when you need to allow members to quickly join/leave the cluster and when consistency is not an issue for you (i.e. you might not see a entry in the cache even if it is there). From the top of my head I can't think about such a scenario though

[shane_dev]

Thanks Mircea,

That is what I figured as well. I just needed to confirm.

Shane