3 Replies Latest reply: Aug 5, 2012 7:06 AM by Vladimir Blagojevic RSS

How to use DistributedExecutorService

John Sanda Novice

I am having some trouble understanding with what types of caches DistributedExecutorService can be used. I previously assumed and tried to use it with a remote cache running in a stand alone hotrod server. I had the following conversation on #infinispan on freenode which now has me a bit confused:


(08:01:10 AM) jsanda: is it possible to distributed executor service with single remote cache? i'm trying in a unit test and hitting, IllegalStateException: Can not use non-clustered cache for DefaultExecutorService

(08:01:21 AM) sannegrinovero: jsanda, you're welcome! looks interesting. Keep in mind you could also use Hibernate OGM for CRUD, but the dist-exec would still be needed as well.

(08:01:39 AM) sannegrinovero: jsanda, no remote caches are more limited. No distributed execution nor queries.


I thought a primary use case for DistributedExecutorService was with distributed, remote caches. I am failing to understand something, just not sure what. How can/should I set up and deploy multiple infinispan nodes with distributed caches so that I can utilize DistributedExecutorService across those nodes?





  • 1. Re: How to use DistributedExecutorService
    Vladimir Blagojevic Master



    DistributedExecutorService works with non-remote caches. I am not sure about your particular requirements but simply create DistributedExecutorService using a particular cache obtained from a CacheManager and invoke distributed tasks on the cluster. Look at the demos if you are still in doubt how to use it!




  • 2. Re: How to use DistributedExecutorService
    jay shaughnessy Expert



    Thanks for the follow up.  Yes, basically what we were asking was whether we could use the DistributedExecutorService with a HotRod server deployment, and it seems that the answer is no.  Because then we're talking about RemoteCaches.  Right?


    For our app (RHQ), we don't think we want to have the RHQ server also be an ISPN node as it would potentially take away from RHQ cycles and also require a cacheStore (we're thinking Jdbm) on the RHQ Server machine. Also, we think we'd like to be able to cycle RHQ servers without actually cycling the ISPN node and incurring state transfer on startup. We are planning to use Distribution mode. This is what had led us towards an ISPN server approach, despite the overhead of the data transfer between client and server.  But we want MapReduce and DistributedCallables.


    Assuming we stick with the RHQ server not being an Infinispan node, the  alernative will likely be to use a P2P deployment of nodes where we "simulate" a client-server deployment.  Probably making each node an AS7 server with a lightweight app that can perform what we need on the local cache and also communicate with the RHQ server in some way to service requests.


    Any thoughts much appreciated.

  • 3. Re: How to use DistributedExecutorService
    Vladimir Blagojevic Master

    No, you are right. No support for distributed execution over HotRod yet. Another thing you might try is to have Infinispan deployed in AS7 and do remote invocation. I've seen examples where people embed Infinispan in AS7, do distributed execution and map/reduce but they invoke it remotely through REST. I suppose you can do remote EJB invocation and such. Experiment and find the best solution for your particular needs. You seem to know what you are doing :-)