Reading the code I found this comment in org.infinispan.interceptors.DistributionInterceptor.visitGetKeyValueCommand (version 5.1.8.Final)
'..Also don't bother with a remote get if
// the entry is mapped to the local node.'
Now consider configuration with number of owners greater than 1. It seems possible to me (and in my environment quite likely), that the entry in local node has been removed (due to cache size eviction), but it's still on another owners node - hasn't been evicted there. In this situation we have only 50% chance, that we will get the element even though it's still available in the cluster.
Is this right, or am I missing something? Can anything be done about this?