Multinode Remote Cluster with 5.3Beta2 Servers
kalinwilson May 27, 2013 7:09 PMTristan, et al,
First, I'm enjoying getting to know Infinispan and I appreciate the documentation on the wiki, although some of the older pages are conflicting with newer documentation. However, I'm surprised there isn't more explicit documentation on setting up a remote cluster/grid, in either a test or operational environment. I expect that this is a very prominent use case for an IMDG, vice local caches clustered by way of multiple process instances, which seems to be the model most covered. Even in Manik's Packt book, the remote case is not covered in any detail. I'm using the wiki page at https://docs.jboss.org/author/display/ISPN/Infinispan+Server, which I see was recently added/updated. It's good about explaining how a single server instance is configured, but doesn't go into how to run several server instances in a cluster (unless I missed it). I'm not familiar with JBOSS AS yet, so I haven't tried to figure out AS clustering.
I'm using bin/clustered.sh to try to set up a 2+ node server cluster on a single machine. When I start the second server I get errors as the protocols try to bind to the same ports the first server is using. I tried using both standalone/configuration/clustered.xml and docs/examples/configs/clustered-two-nodes.xml. Neither would let me simply start multiple instances of the servers, which is what I expect I have to do to get multiple nodes. I looked at the JBOSS AS command line parameters but did not see how to bind hotrod to a different port.
I think I follow the AS configuration for the caches and associated subsystems. Do I need to create config file variants that change the protocol bindings in the <socket-binding-group> element> and use a different file for each node?
*** Would you explicitly lay out the steps for establishing a multinode remote cluster both on a single machine and different machines? Adding this to the wiki would be helpful for a lot of people I bet.
Once I have the cluster up, I expect I can write a client. How do I connect to the cluster with the command line client? I tried ./ispn-cli.sh --connect=//kalin@localhost:11222/clustered/default to connect to the hotrod port with no luck. I also tried jmx://localhost:12000 per the instructions in https://docs.jboss.org/author/display/ISPN/Command-Line+Interface+%28CLI%29.
Additionally, is there a way to remotely inspect the contents of each node of a remote cluster to verify distribution, etc.? Via JMX or the CLI?
I looked at the differences between clustered.xml and clustered-two-nodes.xml and some of the other config files. They seem to be minor. The removal of the modcluster extension, different unicast protocols, the addition of the rsvp protocol, etc. Could you explain how each example config is supposed to be used? A readme in the distro would help a lot.
My evaluation of Infinispan has gone well so far but I really need to verify how a remote cluster is configured, managed, and used. Sorry for the long post but Thanks for any help!
Kalin