-
1. Re: Infinispan 6 - Is Map/Remove consistent?
pruivo Jan 14, 2014 6:20 AM (in response to eskape)Hi Egor,
All your options should work fine. We have never checked for consistency during the map/reduce phase, but it should keep the cache consistency.
However, IMO, the last option is the most related to the map/reduce world.
Cheers,
Pedro
-
2. Re: Infinispan 6 - Is Map/Remove consistent?
eskape Jan 14, 2014 5:09 PM (in response to pruivo)Hi Pedro,
Thanks for swift reply.
I do understand that from theory standpoint the last option is the most "correct" - however, in I also have performance considerations in mind.
My understanding of the process, given distributed cache setup, is:
1) Map/Reduce operation is executed on each node of the cluster;
2) Resulting list is collected on a "master" node (i.e. the one that requested the operation);
3) remove() method is called for each object in the list, and this again gets executed on each node.
If my understanding of the way Infinispan works is adequate, I could save on unnecessary reduce() operation, as well as on collection and re-execution phases if I call evict() method rather than remove() during map() phase because the execution is already distributed.
Please correct me if I'm wrong.
Thanks
Egor.
-
3. Re: Infinispan 6 - Is Map/Remove consistent?
pruivo Jan 15, 2014 3:29 AM (in response to eskape)Hi Egor,
Egor Kolesnikov wrote:
I do understand that from theory standpoint the last option is the most "correct" - however, in I also have performance considerations in mind.
My understanding of the process, given distributed cache setup, is:
1) Map/Reduce operation is executed on each node of the cluster;
2) Resulting list is collected on a "master" node (i.e. the one that requested the operation);
3) remove() method is called for each object in the list, and this again gets executed on each node.
Yes that is correct.
Egor Kolesnikov wrote:
If my understanding of the way Infinispan works is adequate, I could save on unnecessary reduce() operation, as well as on collection and re-execution phases if I call evict() method rather than remove() during map() phase because the execution is already distributed.
Please correct me if I'm wrong.
That works If you have configured numOwners = 1. Otherwise, you will have to use remove(). The map() phase happens in the primary owner of the key and the evict() only removes from local node. So, with evict() and numOwners != 1, the backup owners would never remove the key.
One thing that you may consider is the evict() does not remove the key from CacheLoader/Store (if you have any configured).
Cheers,
Pedro
-
4. Re: Infinispan 6 - Is Map/Remove consistent?
eskape Jan 15, 2014 4:56 AM (in response to pruivo)Hi Pedro
That's a bit unexpected - I was under impression that map() will iterate over each entry in the local cache whether it originated on the node or not. It appears that's not the case.
Thanks for the heads-up, I guess I just go for option 3 then.
Cheers
Egor
-
5. Re: Infinispan 6 - Is Map/Remove consistent?
pruivo Jan 15, 2014 5:01 AM (in response to eskape)Hi Egor,
Why is it unexpected?
If a key K is replicated in N nodes (i.e. numOwners=N), only one node will invoke map() with K, otherwise it would map() N times (that is not desired).
Cheers,
Pedro