I forgot to mention about implementing a custom consistent hashing option - whihc also is equally complicated ...
as far as I know, when you specify numOwners=2 then both nodes are owners of the key.
I'm not sure what you really want to achieve. How many nodes do you have in the data grid overall?
Anyway, IMO you could possibly use numOwners=1 and configure each of the nodes to have RemoteCacheStore (remote connection to another node over HotRod protocol - https://docs.jboss.org/author/display/ISPN/CacheLoaders). Then you would have one group of servers running in distributed mode and a few other nodes running in local mode (not clustered with any other nodes) each of which would be a backup of one node in the main server group.
With this setting you could write keys into the main server group and read from those individual servers.
But this is kind of strange configuration
If i understand right , even though we mention num copies = 2 , i think one of teh nodes is designated as teh owner.
I was basically looking for optmization in locking because i had one node writing and i wanted the lock to be on the local node rather than a remote one.
I think you don't need to worry about performance here.
The single lock owner concept comes handy if you have several transactions *modifying" the same entry. Then you can get deadlocks. But read operations are non-blocking (as described here: https://docs.jboss.org/author/display/ISPN/Locking+and+Concurrency)
Since my business usecase clearly says that - of my two nodes only one nodes writes and teh other one just reads - i do not want to take any overhead of remote locking.
If we do not take care of that and the cache entry is onwed by node 2, Every time node1 tries to modify teh entry , it ends up taking a remote lock. We are just trying to remove teh unecessary remote interctaion when we are very sure of our business usecase. We have quite a lot of such interactions , and we are looking at numebrs around 250 puts per second. So i think this will have a impact on the over all performance of our system.
My experience says that 250 ops per second is fairly low number. If this is all about minimizing lock acquisitions then Optimistic Locking concept should help here. It's always better to measure the performance and optimize afterwards.
We do use optimistic locking. We were just trying to avoid remote calls for lock acquisition when we are sure abt our business usecase, and were looking for options how infinispan supports this.