14 Replies Latest reply on Jan 15, 2008 8:44 AM by bodrin

    unable to choose a default cluster node for messaging

    bodrin

      I want to setup something like a master-slave cluster - all jms clients (external) using only the master node and switch to the slave only if the master is down.

      So, first looking at http://docs.jboss.org/jbossas/jboss4guide/r4/html/cluster.chapt.html

      16.2.2. Client configuration
      
      The JNDI client needs to be aware of the HA-JNDI cluster.
      
      ...
      
      java.naming.provier.url=server1:1100,server2:1100,server3:1100,server4:1100
      
      
      When initialising, the JNP client code will try to get in touch with each server node from the list, one after the other, stopping as soon as one server has been reached. It will then download the HA-JNDI stub from this node.
      Note
      
      There is no load balancing behavior in the JNP client lookup process. It just goes through the provider list and use the first available server. The HA-JNDI provider list only needs to contain a subset of HA-JNDI nodes in the cluster.
      


      I can set it like this:
      java.naming.provider.url=master:1200,slave:1300
      


      Then I lookup the "/ClusteredConnectionFactory" which is configured to not support load-balancing via
      <attribute name="SupportsLoadBalancing">false</attribute>
      

      in JBOSS_CONFIG\deploy\jboss-messaging.sar\connection-factories-service.xml

      I have observed that subsequent createConnection() calls on the ConnectionFactory object chooses one and the same cluster node.

      The problem is that ones it is the master :

      ant
      Buildfile: build.xml
      
      identify:
       [echo] ###########################################################################
       [echo] # Running the DISTRIBUTED QUEUE example #
       [echo] ###########################################################################
       [echo] The queue: testDistributedQueue
       [echo] The client jar: ../..//jboss-messaging-client.jar
      
      sanity-check:
      
      init:
      
      compile:
      
      run:
       [java] Distributed queue /queue/testDistributedQueue exists
       [java] java.lang.RuntimeException: Assertion failed, 0 == 0
       [java] at org.jboss.example.jms.common.ExampleSupport.assertNotEquals(ExampleSupport.java:85)
       [java] at org.jboss.example.jms.distributedqueue.DistributedQueueExample.example(DistributedQueueExample.java:89)
       [java] at org.jboss.example.jms.common.ExampleSupport.run(ExampleSupport.java:147)
       [java] at org.jboss.example.jms.distributedqueue.DistributedQueueExample.main(DistributedQueueExample.java:173)
       [java]
       [java] #####################
       [java] ### FAILURE! ###
       [java] #####################
      
      BUILD FAILED
      


      ... but sometimes it is the slave :

      ant
      Buildfile: build.xml
      
      identify:
       [echo] ###########################################################################
       [echo] # Running the DISTRIBUTED QUEUE example #
       [echo] ###########################################################################
       [echo] The queue: testDistributedQueue
       [echo] The client jar: ../..//jboss-messaging-client.jar
      
      sanity-check:
      
      init:
      
      compile:
      
      run:
       [java] Distributed queue /queue/testDistributedQueue exists
       [java] java.lang.RuntimeException: Assertion failed, 1 == 1
       [java] at org.jboss.example.jms.common.ExampleSupport.assertNotEquals(ExampleSupport.java:85)
       [java] at org.jboss.example.jms.distributedqueue.DistributedQueueExample.example(DistributedQueueExample.java:89)
       [java] at org.jboss.example.jms.common.ExampleSupport.run(ExampleSupport.java:147)
       [java] at org.jboss.example.jms.distributedqueue.DistributedQueueExample.main(DistributedQueueExample.java:173)
       [java]
       [java] #####################
       [java] ### FAILURE! ###
       [java] #####################
      
      BUILD FAILED
      


      Do you know if it is possible to make such a setup that it will always choose the master (serviceID=0) node?
      I mean the one where the ha-jndi stub is connected to (There is no load balancing behavior in the JNP client lookup process.) ?

      Note that I have changed the distributed-queue example - I have added an InitialContext configuration:
      ...
       // connecting to the first node
       java.util.Hashtable properties = new java.util.Hashtable();
       properties.put(javax.naming.Context.INITIAL_CONTEXT_FACTORY,
       "org.jnp.interfaces.NamingContextFactory");
       properties.put(javax.naming.Context.URL_PKG_PREFIXES,
       "org.jboss.naming:org.jnp.interfaces");
       properties.put(javax.naming.Context.PROVIDER_URL, "localhost:1200,localhost:1300");
      
       ic = new InitialContext(properties);
      ...
      



        • 1. Re: unable to choose a default cluster node for messaging
          timfox

           

          "bodrin" wrote:
          I want to setup something like a master-slave cluster - all jms clients (external) using only the master node and switch to the slave only if the master is down.


          If you want a JBoss MQ style approach where only one node is active at any one time, you can deploy JBM as a HA Singleton - there is a FAQ on this.

          • 2. Re: unable to choose a default cluster node for messaging
            bodrin

            Ok. But here http://labs.jboss.com/jbossmessaging/ you say that :

            ...
            JBoss MQ is currently in bug fix mode only, and all new messaging development is channelled towards JBoss Messaging.
            ...
            

            ...
            and
            ...
            ...
            The goal of JBM 2.0 is simple and uncompromising- to be the fastest and most scalable reliable messaging system in the open source space, without exception.
            ...
            


            So, I do not want JbossMQ :)

            Don't you think that when I configure the ClusterConnectionFactory to not load-balance it should use only the local node - the one to which I have a connecttion already?

            • 3. Re: unable to choose a default cluster node for messaging
              timfox

               

              "bodrin" wrote:


              Don't you think that when I configure the ClusterConnectionFactory to not load-balance it should use only the local node - the one to which I have a connecttion already?



              if you set supportsLoadBalancing to false on the ConnectionFactory then all connections created using that connection factory will be to the node from which you obtained that connection factory.

              I.e. it doesn't load balance.

              Is that what you want?

              • 4. Re: unable to choose a default cluster node for messaging
                bodrin

                Thank you for the fast reply! And sorry - I though that I have to use JbossMQ...

                I have found some related topics:

                http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4104294#4104294

                http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4104715#4104715

                I'm not sure yet if I'll need to preserve the order in case of one consumer for some queue.

                But I want a cluster just for the failover. I want all my producers and consumers connected to one node - the master one, the slave is just to replace it in case of failure.

                Also I think that all the messaging will work faster if there is no pulling of messages from one node to another.

                Can you tell me where can I find this FAQ of how to configure JBM as a ha-singleton?

                • 5. Re: unable to choose a default cluster node for messaging
                  timfox

                   

                  "bodrin" wrote:

                  Can you tell me where can I find this FAQ of how to configure JBM as a ha-singleton?


                  With all the other FAQs on the wiki page ;)

                  I'll leave it as an exercise for you to find the JBoss Messaging wiki page...

                  • 6. Re: unable to choose a default cluster node for messaging
                    bodrin

                    Ok. Here it is : http://wiki.jboss.org/wiki/Wiki.jsp?page=JBMHASingleton

                    but it will not work for me, becuase of this:

                    ...
                    When running this way you will lose other benefits of JBoss Messaging like automatic failover and distributed destinations.
                    ...
                    


                    Anyway, I can not undestand the behaviour of the current JBM. I'm connecting to the master node (ha-jndi) and I have disabled the load balancing of the ClusteredConnectionFactory and sometimes I get
                    Assertion failed, 0 == 0
                    sometimes
                    Assertion failed, 1 == 1

                    I think i should alway get : Assertion failed, 0 == 0

                    • 7. Re: unable to choose a default cluster node for messaging
                      timfox

                       

                      "bodrin" wrote:

                      Anyway, I can not undestand the behaviour of the current JBM. I'm connecting to the master node (ha-jndi) and I have disabled the load balancing of the ClusteredConnectionFactory and sometimes I get
                      Assertion failed, 0 == 0
                      sometimes
                      Assertion failed, 1 == 1

                      I think i should alway get : Assertion failed, 0 == 0


                      I suspect you are not using the same connection factory instance.

                      With a single connection factory instance that has supportsLoadBalancing = false, it will always create connections to the same node.

                      You are probably downloading different connection factories from different nodes.

                      • 8. Re: unable to choose a default cluster node for messaging
                        bodrin

                        Ok, so you mean that this code:

                         java.util.Hashtable properties = new java.util.Hashtable();
                         properties.put(javax.naming.Context.INITIAL_CONTEXT_FACTORY,
                         "org.jnp.interfaces.NamingContextFactory");
                         properties.put(javax.naming.Context.URL_PKG_PREFIXES,
                         "org.jboss.naming:org.jnp.interfaces");
                         properties.put(javax.naming.Context.PROVIDER_URL, "localhost:1200,localhost:1300");
                        
                         ic = new InitialContext(properties);
                        
                         cf = (ConnectionFactory)ic.lookup("/ClusteredConnectionFactory");
                        


                        sometimes obtains the ClusteredConnectionFactory instance from one cluster node and sometimes from another one?

                        Is this because of the HA-JNDI behaviour?

                        If this is right - is there a way to choose always the node at localhost:1200?

                        • 9. Re: unable to choose a default cluster node for messaging
                          timfox

                          Bodrin-

                          I am trying to understand why you only want one node in the cluster to be active at any one time?

                          Why not let your clients connect to any node, then you won't have these issues.

                          Maybe I don't understand your motivations for doing this...

                          • 10. Re: unable to choose a default cluster node for messaging
                            bodrin

                            Imagine you have designated a powerfull hardware box for your messaging system only - the master node. Just to ensure that when something goes wrong with this box you want your clients still to be able to send messages you are running a cluster. And you have no enough money to buy the same kind of box and you get a smaller one - the slave. It is there only for the failover support, so you want all consumers and all producers to use the master node by default.

                            • 11. Re: unable to choose a default cluster node for messaging
                              timfox

                              Ok, you just described the HA singleton.

                              JBM supports this mode too, as mentioned before: http://wiki.jboss.org/wiki/Wiki.jsp?page=JBMHASingleton

                              HTH

                              • 12. Re: unable to choose a default cluster node for messaging
                                bodrin

                                OK, I see..

                                The bad thing is that :

                                ...
                                When running this way you will lose other benefits of JBoss Messaging like automatic failover and distributed destinations.
                                ...


                                But anyway I have to survive a cluster restarts also.
                                So I suppose the only way to survive cluster restart even in the clustered messaging (not ha-singleton mode) is to use the connection.setExceptionListener(...) , right ?

                                • 13. Re: unable to choose a default cluster node for messaging
                                  timfox

                                   

                                  "bodrin" wrote:
                                  OK, I see..

                                  The bad thing is that :

                                  ...
                                  When running this way you will lose other benefits of JBoss Messaging like automatic failover and distributed destinations.



                                  Well distributed destinations are meaningless anyway if you only have one active node.


                                  But anyway I have to survive a cluster restarts also.
                                  So I suppose the only way to survive cluster restart even in the clustered messaging (not ha-singleton mode) is to use the connection.setExceptionListener(...) , right ?


                                  That's right.

                                  • 14. Re: unable to choose a default cluster node for messaging
                                    bodrin

                                    Ok. So, forget about the ha-singleton mode.

                                    Is there a way to keep the transparent/automatic client failover and to handle only the cluster restarts via connection.setExceptionListener(...) ?