You would need to use synchronous replication for such guarantees. A call - e.g., cache.put() - will not return until the same call is successfully executed across the cluster. And if a remote instance is attempting the same thing, it will not complete until the call can be executed across the cluster as well. As such, both calls will block until one of them times out, causing one of the calls to fail and the other to succeed.
In the same node if Host A updates key value "AAA" to "1" and Host B simultaneously updated key value "AAA" to "2" would Host A's value be "2" and B's be "1" after replication? The caches would become out of sync?
So the solution is to use sync cache or switch to a INVALIDATION_ASYNC cache to inform both Hosts that they should go to other persistence to get the "real" value?
For a write heavy scenario like a shared counter, yes.