10 Replies Latest reply on Apr 19, 2007 7:28 AM by timfox

    Failover scenario is not working

    sabdulkareem

      Hi all,


      I am trying to setup failover for our standalone applications(MessageSender and MessageReceiver) using JBoss Messaging.

      I have created 3 nodes in my local machine to form a cluster like:
      Message-Node0, Message-Node1,Message-Node2 and started the three nodes.

      This is the ConnectionFactory properties I used for the Sender and Receiver applications:

      jmsConnectionFactory.jndiInitialContextFactory=org.jnp.interfaces.NamingContextFactory
      jmsConnectionFactory.jndiProviderUrl=jnp://localhost:1099
      jmsConnectionFactory.jndiNameForJmsConnectionFactory=ConnectionFactory

      I tested the following scenario:

      1. Started the Receiver application (It is a 24x7 application)
      2. Started the Sender application which sends text messages to the testDistributedQueue.
      3. While publishing some messages I stopped the Messaging-node0 server.
      4. The Sender and Receiver applications tries to get Connections using Recovery Algorithms.

      I expected the messages will be published by getting connection(auto-discovery) using the other available nodes in the cluster(ie, node1 and node2)
      But this is not happening.

      This is because the application fails to discover the other nodes in the cluster.( We tried the same failover scenario for JbossMQ, that works fine.)

      The application throws the following exception:


      javax.naming.CommunicationException: Could not obtain connection to any of these urls: localhost:1099 and discovery failed with error: javax.naming.CommunicationException: Failed to connect to server 10.212.65.25:1100 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server 10.212.65.25:1100 [Root exception is java.net.ConnectException: Connection timed out: connect]] [Root exception is javax.naming.CommunicationException: Failed to connect to server localhost:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server localhost:1099 [Root exception is java.net.ConnectException: Connection refused: connect]]]



      If I use the port 1100 instead of 1099 the applications hanging (ie, the application does not throws any exception and also not publishing and receiving messages)

      Do I need to setup anything to enable HAJNDI for each instance? Pls help.

      Version Details:

      Jboss 4.0.5
      Jboss Messaging 1.2.0 sp1



      Thanks in Advance!

        • 1. Re: Failover scenario is not working
          timfox

          Do the clustering examples run?

          • 2. Re: Failover scenario is not working
            timfox

             

            "sabdulkareem" wrote:

            4. The Sender and Receiver applications tries to get Connections using Recovery Algorithms.


            JBM uses transparent failover so there is no need to recover the connections yourself, i.e. you don't need to implement an Exception Listener.

            When failover occurs the connections should automatically be failed over onto a different server.

            Have a look at the clustering examples that come with the distro.

            • 3. Re: Failover scenario is not working
              sabdulkareem


              I tried the examples distributed-queue, queue-failover.
              Both are working fine.

              In queue-failover example the application kills the Messaging-node0 programatically. But I am using Ctrl+C to kill the Messaging-node0.

              Will it make any difference?

              Thanks & Regards,
              Abdul Kareem




              • 4. Re: Failover scenario is not working
                timfox

                 

                "sabdulkareem" wrote:


                Will it make any difference?



                Yes, CTRL-C does not kill the server - it performs a clean shut down. Clean shut down and crash are different things.

                There is also a bug in jboss remoting that prevents reconnections in some situations - this has been fixed in remoting 2.2.0.SP1.

                • 5. Re: Failover scenario is not working
                  sabdulkareem

                  Thanks a lot!

                  I have replaced the jboss-remoting related jars to the version2.2.0 in my application. Now the Sender application able to get Connection on Node1, if the Node0 fails. But the Receiver application throws the exception like:


                  2007-04-19 15:20:44,861 [main] WARN - Application re-initialization attempted
                  Got Message: [File1 - Message1File1 - Message2File1 - Message3File1 - Message4]
                  2007-04-19 15:21:12,059 [Timer-1] WARN -
                  org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@1342a67: detected
                  failure on control connection Thread[control: Socket[addr=ddp4344.cptdomain.com/192.168.51.48,port=3197,
                  localport=2112],5,]: requesting new control connection
                  2007-04-19 15:22:03,914 [Thread-8] WARN - Client attempted failover,
                  but no failover attempt has been detected on the server side.
                  We will now try again on the original server in case there was a
                  temporary glitch on the client--server network
                  2007-04-19 15:22:15,900 [Thread-8] ERROR - Unable to retrieve address/port of secondary server socket
                  org.jboss.remoting.CannotConnectException: Can not get connection to server.
                  Problem establishing socket connection for InvokerLocator
                  [bisocket://192.168.51.48:4457/?NumberOfCallRetries=2&clientMaxPoolSize=50&clientSocketClass=
                  org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=
                  org.jboss.jms.server.remoting.JMSWireFormat&numberOfRetries=1&serializationtype=
                  jms&socket.check_connection=false&unmarshaller=org.jboss.jms.server.remoting.JMSWireFormat]


                  What could be the reason?

                  Please Suggest me.

                  Thanks & Regards,
                  Abdul Kareem

                  • 6. Re: Failover scenario is not working
                    timfox

                    Please can you explain again clearly exactly what your set up is and what you are doing (killing process control-c or whatever)?

                    Also can you post your code example?

                    Thx

                    • 7. Re: Failover scenario is not working
                      timfox

                      Also - you said version 2.2.0 - you meant 2.2.0.SP1?

                      • 8. Re: Failover scenario is not working
                        sabdulkareem


                        I have used JBossRemoting-2_2_0_GA for my application. Can you please provide me the link to download JBossRemoting-2_2_0 sp1?

                        Thanks & Regards,
                        Abdul Kareem

                        • 9. Re: Failover scenario is not working
                          timfox
                          • 10. Re: Failover scenario is not working
                            timfox