Using DistributedCallable for search/filtering

[srininara]

I am trying to use Distributed Callable approach for doing search/filtering for my Infinispan Proof of Concept.

I will provide an example:

I have Person objects stored across the grid and I want to find number of Person objects which hail from a specific city (assuming city is a attribute of Person). So I create a DistributedCallable filter. The filter object is sent across to all nodes. Now since I have to search across all the Person objects in the system, in each node I get the cache.keyset() and do the filtering to get the count. The keyset() ofcourse gives me a set of local entries (which would contain duplicates because I have a replication factor of two). Now from the outputs of these callables, I do an addition and then divide it by number of replicas (something close to that).

While this is doable, it is not very nice. First I am using keyset() which is considered a bad practice in production (according to the javadoc). Also the division of number of replicas seems non-intuitive. Is there a better way?

I initially tried using hibernate/lucene query based stuff but somehow the setting up the index was complex (it always gave me wrong results) and hence I tried using the DIstributedCallable approach.

Any help will be appreciated.

Thanks

Best Regards

Srini

[vblagojevic]

Do a map/reduce on your Infinispan cluster. Each time you find a person from a certain city emit count 1. In reducer just add them up. See WordCount map/reduce demo in demos module for inspiration and further guidance.

Regards,

Vladimir

[srininara]

Thanks Vladimir for the response. Based on your input, I did some quick read through of the MapReduceTask and supporting classes code from Github. The MapReduceManagerImpl internally seems to be using cache.keyset() and DistributionManager related methods to find out whether the key is local or not. I think I can do the same thing in my DistributedCallable. Let me know if you have any conerns. I will make changes and try it out and post the code as soon as possible.

Thanks,

Srini

[vblagojevic]

Yes, calling cache#keySet() from DistributeCallable should possibly do the trick. Keep us updated about your progress.

Regards,

Vladimir

[srininara]

I was able to do this. Please find the code at my github repo.

I could not use the direct method in DistributionManger.getLocality(key).isLocal() - it kept returning true always. So I had to compare addresses. That worked

On a related note, I was trying to figure out a way to achieve pagination. For that it seems I have to use Hibernate query only. If there is a different way, please do let me know. If you want me to start a different thread for this, please let me know and I will immediately do that.

Thanks,

Srini