With distributed mode , will local copy get preference over remote (with consistent hashing)

[sudheerk84]

I have a use case where i plan to use 'distributed mode' with number of copies = 2.

Under such case , can i be assured that if any request going to these two nodes teh get for the cache making a local call instead of remote call ?

[To clarify , can any of these nodes make a call to the other inspite of having a local copy since it is using consistent hashing to determine teh node]

[galder.zamarreno]

If the copy key resolves locally, the local value will be used.

[sudheerk84]

Does it mean that i have use soem key affinity service to resolve teh key locally OR is there is a better and simpler way to do this ?

[My use case : - only one node writes, and one other node reades (so one node writes , two nodes read ., none of the other nodes need this data) ].

I want to get local read in both these nodes. (one which writes and the one which reads)

[galder.zamarreno]

Local reads will be achieved by enabling L1 caching in distribution. What L1 does is that if a key is requested from node X and it that does not belong to node X, once it's retrieved it from another node, it will cache for a period of time, so reads are local. If the key belongs to this node, it will already be available locally. More info in https://docs.jboss.org/author/x/GwY5

[sudheerk84]

Really nice to knwo that. I have gone through the documentation of L1 cache . But none of them clarify the case when "key is already present in teh node before teh first remote call". Adding this to documentation will benefit others as well.

So if i understand right,  "When i have distributed caching enabled , with num copies = 2 , and L1 cache enabled in both nodes, every read will be local ".

I was of teh impression that L1 cache is a seperate copy of the items ?  Isnt it so ?

Will there be extra invaliation messages ?  [In teh case above there is no need of invalidation messages since cache copies will anyways be updated as part of distributed mode of caching on these two nodes ]

[galder.zamarreno]

Really nice to knwo that. I have gone through the documentation of L1 cache . But none of them clarify the case when "key is already present in teh node before teh first remote call". Adding this to documentation will benefit others as well.

Well, this is how internals work really. It's really an optimisation. This is an open source project and you can trace the code to see what happens

So if i understand right,  "When i have distributed caching enabled , with num copies = 2 , and L1 cache enabled in both nodes, every read will be local ".

Well, depends how many nodes you have. If you only have 2 nodes, L1 is useless, cos having 2 number of copies, all nodes will have the data. That's effectively total replication. With more nodes, even with L1, if the data is not present locally, it'll go remote. But with L1, reads after that will be local. Without L1, reads for data not present locally will always be remote.

I was of teh impression that L1 cache is a seperate copy of the items ?  Isnt it so ?

It's not, read the code

Will there be extra invaliation messages ?  [In teh case above there is no need of invalidation messages since cache copies will anyways be updated as part of distributed mode of caching on these two nodes ]

If you're really going to set up distribution with 2 copies and only two nodes in the cluster, it sounds like a rather pointless set up. Might as well rely on replication which is simpler code base.

Anyway, of couse, with L1 enabled, when data is modified, invalidation messages will be sent around. Granted, if two copies and two nodes, this is useless, but the whole set up is pointless from the start

[sudheerk84]

Thanks for teh response.

I have around 10 nodes in my setup.

9 nodes of which are app engine which server different kind of product . (Node1 servers product1 hence has cache1 , node 2 serves product2 has cache2 , ............. node9 serves product9 has cache 9) .  

10th node is a risk engine which needs data from all these 9 nodes(cache1 , cache2 , ...cache9) and does only  reading from these caches. This node doesnt write into any cache.

Hence

cache1 - written by node1 , read by node1 and node 10

cache 2 - written by node 2 , read by node 2 and node 10

.......

cache 9 - writeen by node 9 , read by node 9 and node 10

So what i am essentially trying  to achive is

1. Cache replication is minimal

2. All reads are local only

3. no locking happens in reading nodes (because only node writes)

Hence i do not want to use replicated mode.

So i was thiking of uisng distributed mode, with num copies = 2 and mentioning different rack names for (node 1 to node 9 (rack 1)  && node 10 (rack 2) = so that i will get teh above defined functionality)

Basically what i am looking at is buddy replication between node1 and node 10 , node 2 and node 10, node 3 and node 10 ....... until node 9 and node 10. Also with optimization that when node 1 to node 9 writes i want only local locking.

Is there a better distribution mechanish to achive the same ?

And the reason for me to think about using L1 caching  is - since distributed mode uses consistent hasing - though cache1  is replicated in node 1 and node 10 as explained in scenario above , when one teh nodes read , i though it will make a remote call (sicne the consistent hash may be show teh other node as owner).

[galder.zamarreno]

That looks Ok to me. If node 1 is never gonna need to read cache2-9, then L1 would not be needed.

[sudheerk84]

Thanks for the confirmation

Also to ensure there is minimal locking when node 1 (or node 2 .or nde 3 or ....) writes , i was thinking of enabling USEEAGERLOCKING and  EAGERLOCKSINGLENODE.

Hence i started teh thread on the same - https://community.jboss.org/message/718402#718402

Basically i just wanted node 1(node 2 , node 3 , ....) to take local local , and not try to get a remote lock on node 10 , when i commit the transaction.

[galder.zamarreno]

Those options make most sense when contention is expected on a key. That doesn't seem to be your case.

[sudheerk84]

I am a bit confused ..

If EAGERLOCKSINGLENODE  - doesnt mean do not lock other nodes ,

What is teh right flag one to use in my use case, where i want lock only on node 1 and not on node 10 when i am writing and commiting a transaction on cache 1.

In replicated mode - i see locks are taken on all nodes when i commit transatcion . I thought it was the same case in dist(locks taken on number of copies ). So does this mean  is dist mode when ever i write it takes a lock on only on single node .

[galder.zamarreno]

I am a bit confused ..

If EAGERLOCKSINGLENODE  - doesnt mean do not lock other nodes ,

What is teh right flag one to use in my use case, where i want lock only on node 1 and not on node 10 when i am writing and commiting a transaction on cache 1.

In replicated mode - i see locks are taken on all nodes when i commit transatcion . I thought it was the same case in dist(locks taken on number of copies ). So does this mean  is dist mode when ever i write it takes a lock on only on single node .

That should not be the case from Infinispan 5.1.0.FINAL because it only acquires locks in a single node for both replicated and distributed caches, hence, eagerLockSingleNode has now been replicated.

See http://docs.jboss.org/infinispan/5.1/configdocs/urn_infinispan_config_5.1/complexType/configuration.transaction.html#attr_eagerLockSingleNode