2 Replies Latest reply on Apr 11, 2008 3:58 AM by timfox

    JBM clustered connection factory and load balancing

    meyers79

      We have been struggling with this for some time and I am hoping that someone can help shed some light.

      Are current architecture is as follows:

      A clustered pair of JBA 4.2 servers serving up UI code

      * Publishers on the UI nodes post messages to be picked up by batch nodes

      A clustered pair of JBM 1.4 servers

      * hosting 1 clustered queue


      A clustered pair of JBA 4.2 servers serving as backend processing servers (batch nodes).

      * Consumers (MDB'S) listen on remote queue to receive messages from the UI nodes.

      Currently using the clustered connection factory (load balancing set true) to get a connection and post messages to the messaging nodes, we were expecting to see the messages sent to the queue from the UI nodes to be evenly distributed across both JBM nodes in a round robin fashion.



      What we have observed

      1. Messages appear to be round robin when both messaging nodes have an equal amount of Consumers.

      2. If one node has no Consumers it appears messages are not sent to that node.



      (No problems here assuming consumers get balanced)



      When subscribing we use MDB's that also use the clustered connection factory, and expect that consumers will balanced across the

      Cluster in a round robin-fashion.



      1. The behavior here appears completely inconsistent. We have run many test cases and have not seen this round-robin.

      Most of the time we will have two consumers land on the same node, and no consumers on the other.



      2. With consumers not evenly distributed it appears the messages all get sent to the one node with consumers attached

      and thus load is not evenly distributed (1 messaging server doing all the work).



      3. It is also observed that not both consumers process incoming messages. Once we bring down consumers which was currently active, Inactive consumer will start picking up message after certain time delay.



      Note that we use clustered Connection Factory and lookup using JNDI lookup specifying java.naming.provider.url as jnp://msgNode1:1100, jnp://msgNode2:1100 where msgNode1 and msgNode2 are IPs of two messaging nodes part of same message cluster



      Is this expected behavior? Are others running this type of configuration successfully?



      Per the comments in the previous post it appears most run there MDB's in the same JVM as the JMS provider.

      This is not the case in our implementation, and I have struggled trying to find a working example of this.

        • 1. Re: JBM clustered connection factory and load balancing
          jmesnil

           

          "Meyers79" wrote:
          We have been struggling with this for some time and I am hoping that someone can help shed some light.

          ...

          Currently using the clustered connection factory (load balancing set true) to get a connection and post messages to the messaging nodes, we were expecting to see the messages sent to the queue from the UI nodes to be evenly distributed across both JBM nodes in a round robin fashion.


          The load balancing is not a true round robin: the local node is always favored if it can consume messages.
          This avoids unnecessary network hops to send the message to another node if it can be consumed locally:

          http://www.jboss.com/index.html?module=bb&op=viewtopic&t=133370

          hope it helps,
          jeff

          • 2. Re: JBM clustered connection factory and load balancing
            timfox

             

            "jmesnil" wrote:

            The load balancing is not a true round robin: the local node is always favored if it can consume messages.
            This avoids unnecessary network hops to send the message to another node if it can be consumed locally:

            http://www.jboss.com/index.html?module=bb&op=viewtopic&t=133370

            hope it helps,
            jeff


            We should be a bit careful about what terminology we use here.

            Normally in JBM when we refer to "load balancing", we're referring to the load balancing of create connection requests across nodes of the cluster. I.e subsequent connections will be created on different nodes of the cluster - this is true round robin load balancing, and is enabled by the "supportsLoadbalancing" attribute on the connection factory.

            Then there is another type of "load balancing" - but in JBM we call this "message redistribution" to avoid confusion.

            Message redistribution is the movement of messages which are already in queues between nodes of the cluster according to load.