How to dedicate nodes in a compute grid.

[greyfairer2]

Hi all,

We want to use Infinispan as a compute grid. We found the documentation on the Distributed Execution Frameworkin Infinispan 5.0: https://docs.jboss.org/author/display/ISPN/Infinispan+Distributed+Execution+Framework.

What we want to do is to dedicate some nodes of the cache as dedicated nodes for executing particular tasks, since only these nodes have the necessary hardware.

My idea was to create a distributed cache mapping HardwareDriverKey to HardwareDriver, and execute the task using

DistributedExecutorService.submit(task, hardwareDriverKey).

For this to work, we need to figure out a way to ensure that the hardwareDriverKey is always located on the particular node of the distributed cache containing the actual hardware.

Do we need to write a custom ConsistentHash that can extract the node address from the hardwareDriverKey? Have you got an example for this? Or is there another way?

Thanks in advance,

Geert.

[ckulenk]

I think this is a good solution. You can wrap another ConsistentHash that handles any other keys. Maybe you can use the TopologyAwareAddress class with a special MachineId format to identify nodes with special hardware profiles.

[matlach]

Hello Geert,

I'm in the same situation. I currently have a custom event system that dispatch cluster event through RMI ; some events need to be broadcasted onto all nodes, while some others only sent onto a specific node.

For the broadcast scenario, I would rely onto submitEverywhere(...) method, but then, how I manage to send an event onto a well defined node ?

I would really like to get rid of RMI and get my event go through the infinispan distributed executor framework.

To achieve this however, it seems, the only way of doing it is to rely on key locality.

Did you find anything yet to implement it ?

Thanks a lot,

Mat.

[ckulenk]

How do you determine the specific nodes? I would love to see a simple Pub/Sub implementation for Infinispan.

I use a custom ConsistentHash for a university project:

//in ConsistentHash impl
public List<Address> locate(Object key, int replCount) {
    if (key instanceof ToAddress) {
        return ((ToAddress) key).getAddresses();
    }
    //...
    return childCh.locate(key, replCount);
}

public interface ToAddress {
    List<Address> getAddresses();   
}

Custom commands or modifying the ExecutorService might lead to a cleaner implementation. In my opinion going away from keys and key locality breaks the idea of infinispan and similar distributed systems. The problem is how to implement complex key-node-mappings (injecting data into a custom ConsistentHash implementation might lead to inconsistencies...).