I have an issue with the way the hotrod client picks the server.

What I want to do is use the hotrod client to do failover only.

i.e. the client picks one server to talk to and only moves to another server when that one fails.

The reason is a basic scalability problem. Currently the hotrod client by default load balances requests across all servers. So if you have say 100 clients using 5 ispn servers you need 100 open tcp/ip sockets per server (ignoring connection pooling on the client which would make it worse) and 5 on the client. If your load doubles, i.e. 200 clients with 10 ispn servers then it is 200 per server and 10 per client.

You should be able to see that this will eventually use up valuable sockets, file descriptors and threads on either the server or client as you try to scale out. If however you just keep it to 1 server per client, you can scale them linearly together.

Obviously the traffic between the servers will increase as you add more servers but you can configure it to cache things of interest to that client locally so it only has to do remoting calls for writes/invalidations.

But, it is actually impossible to stop it doing this, by for example writing your own RequestBalancingStrategy that tries to keep clients sticky to servers. That policy is not always used if the operation extends AbstractKeyOperation which is most of the interesting ones. :-)

https://github.com/infinispan/infinispan/blob/master/client/hotrod-client/src/main/java/org/infinispan/client/hotrod/impl/operations/AbstractKeyOperation.java

Instead it uses the key's hash to determine the server not the balancing, requiring all clients to connect to all servers.