0 Replies Latest reply on Jun 6, 2014 4:43 AM by joc

    Can you have multiple subsystems sharing a jgroups stack?

    joc
    joc Jun 6, 2014 4:43 AM

      We have an application running on JBoss EAP 6.3.0.Alpha1, we just added some code that makes use of infinispan, so I have been experimenting with trying to get both Infinispan and HornetQ playing nice together and sharing a JGroups stack so we only need to configure a single set of ports. I failed.

      I'm not even sure this is or should be possible, and is maybe just down to me missunderstanding the documentation..


      It started with the JGroups documentation, and the section about Sharing a transport between multiple channels in a JVM. That got me thinking this might be possible. The diagram of a shared transport seems to indicate that you can, but I could only get it working with 2 separate stacks using different ports.

      Infinispan seems to have a lot more control here, but messaging doesn't seem to be enough options available to you thru a standalone xml type configuration to be able to achieve it, but I dont clearly understand what I'm doing and what the logs are actually telling me.

       

      There is no great need for this, it's just partly a learning exercise, and an attempt to minimise port usage and configuration needed.


      If it is possible, below are a collection of things I have tried so you can see how far off the mark my thinking might have been.


      Using a standard standalone-ha.xml as the starting point, I modified the messaging subsystem to use jgroups :-

       

      <subsystem xmlns="urn:jboss:domain:messaging:1.4">
<...>
<broadcast-groups>
     <broadcast-group name="AdaptBroadcastGroup">
          <jgroups-stack>tcp</jgroups-stack>
          <jgroups-channel>hq-cluster</jgroups-channel>
<...>
<cluster-connections>
     <cluster-connection name="AdaptCluster">
          <address>jms</address>
          <discovery-group-ref discovery-group-name="AdaptDiscoveryGroup"/>
<...>

      This gives me this in the output log

       

      15:56:46,924 INFO  [stdout] (ServerService Thread Pool -- 65) GMS: address=bis01-0724/hq-cluster, cluster=hq-cluster, physical address=172.16.50.120:7600


      Not sure what this means, but it looks like it's using the channel name for the cluster too, but there doesnt seem to be a way of changing this, so run with it.

      When I add this :-

       

      <subsystem xmlns="urn:jboss:domain:infinispan:1.5">
 <cache-container name="systemCache" default-cache="repl">
  <transport stack="tcp"/>
  <distributed-cache name="settings" mode="ASYNC" batching="false">
   <file-store/>
  </distributed-cache>
 </cache-container>

       

      I get the following errors:-

       

      16:31:54,319 WARN  [org.jgroups.protocols.MPING] (MPING) bis01-0724/hq-cluster: discarding discovery request for cluster 'systemCache' from bis01-0724/systemCache; our cluster name is 'hq-cluster'. Please separate your clusters cleanly.
16:32:14,082 WARN  [org.jgroups.protocols.MPING] (MPING) bis01-0724/systemCache: discarding discovery request for cluster 'hq-cluster' from bis01-0724/hq-cluster; our cluster name is 'systemCache'. Please separate your clusters cleanly.


      Not entirely sure what the separate your clusters cleanly means, but from a couple of forum posts it looks like each cluster must have its own port? So I looked for a way to share the cluster.


      I tried adding shared to the transport as below, but that didnt seem to change anything.

       

      <subsystem xmlns="urn:jboss:domain:jgroups:1.1">
 <stack name="tcp">
  <transport type="TCP" socket-binding="jgroups-tcp" shared="true"/>
<...>


      Tried adding a cluster to the infinispan settings to nudge it towards using the same one,

       

      <subsystem xmlns="urn:jboss:domain:infinispan:1.5">
 <cache-container name="systemCache" default-cache="repl">
  <transport stack="tcp" cluster="hq-cluster"/>
<...>


      but then I get a different error

       

      17:18:00,297 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/JMS_WEB].[joc.infinispantest.TestCache]] (http-BIS01-0724/172.16.50.120:8080-8) JBWEB000235: Allocate exception for servlet joc.infinispantest.TestCache: java.lang.IllegalStateException: cluster 'hq-cluster' is already connected to singleton transport: [dummy-1401985034067, hq-cluster]

      No idea what the dummy bit is refering to here or where it came from.


      Beyond that, I tried naming everything the same as the stack name, multiple stacks with different names, and a number of different variations on a theme - but nothing seemed to work.

       

      So is it simply just not possible, and I'm really getting it all messed up and you can not have multiple things over the same ports?