9 Replies Latest reply on Oct 17, 2012 2:56 AM by Michael Woldrich

    MDB reconnect fails silently

    Michael Woldrich Newbie

      Hello.

      The problem are relating to: https://issues.jboss.org/browse/HORNETQ-981#comment-12723104. But we do not remove any journal. We use the same MDB

       

      Testcase: 2 JBoss-6.0.0 instances. 1st instance with HornetQ-2.1.2. 2nd JBoss instance without HornetQ and with MDB. We killing 1st instance and after startup, the MDB didn't receive messages anymore.

      We get an error, the only error, on 1st instance with: HornetQException[errorCode=100 message=Queue jms.queue.testQueue does not exist].

      The problem I think is that reconnection mechanism don't wait for the resource (queue) MDB needs. Reconnection aborts when queue is not available and didn't try again. Take a look on class ClientSessionImpl#handleFailover

      I also find out that the error message only occures, when a JBoss instance needs more time to boot (our productive JBoss) and therefor more time to deploy his resources. Tests with an fresh JBoss runs always successful.

       

      I also think there are a relation to Deployment ordering on HornetQ destinations and MDBs https://community.jboss.org/message/544222#544222

       

      Please tell me if there are already an fix or is there another way to fix it?

       

      Greetings

        • 1. Re: MDB reconnect fails silently
          Justin Bertram Master

          Couple of things...

           

          1. Reproduce this issue with at least HornetQ 2.2.14.
          2. Provide logs from the server and client instances.
          • 2. Re: MDB reconnect fails silently
            Michael Woldrich Newbie

            Hi.

            I will check this.

            But I hope there are not much configuration changes between 2.1.2 and 2.2.14.

            Are there any integration documentation into JBoss-6.

            Because the integration of a new HornetQ version into our framework, seems to be a lot of work and we already use it poductive.

            • 3. Re: MDB reconnect fails silently
              Justin Bertram Master

              At the end of the day, community support for old versions is simply untenable.  There just isn't enough time or resources.

               

              If you need support and you don't have a strategy to keep your environment on the latest HornetQ release then I recommend you look at an enterprise support contract from Red Hat.  Long term support (including patches) is one the main benefits of such a contract.

              • 4. Re: MDB reconnect fails silently
                Michael Woldrich Newbie

                Hello, I test it with HornetQ version 2.2.14, but we get the same problem. I'm using the documentation for updating hornetq: https://community.jboss.org/thread/169780

                 

                I attach  log files and test programs.

                 

                Test case was:

                1. starting  jboss1 with HornetQ server

                2. starting jboss2. Client consumer is connecting via JNDI

                3. starting a producer which sends messages to a testQueue

                4. checking if messages received on client jboss2 instance

                5. kill JBoss1 and restart it          

                6. the producer sends messages again, if jboss1 is available

                --> We expected that MDB reconnects to the queue, but it never gets messages.

                 

                only if we undeploy consumer, messages are received again

                 

                In 5 we get Error:

                     HornetQException[errorCode=100 message=Queue jms.queue.testQueue does not exist]

                         at org.hornetq.core.server.impl.ServerSessionImpl.createConsumer(ServerSessionImpl.java:339)

                         at org.hornetq.core.protocol.core.ServerSessionPacketHandler.handlePacket(ServerSessionPacketHandler.java:214)

                         at org.hornetq.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:508)

                          ...

                before queues are created

                 

                Again, we get this problem on our productive environment, which needs more time to deploy all resources. Client reconnection is trying to connect to a queue which are currently not exists and it never tries again.

                 

                Greetings

                • 5. Re: MDB reconnect fails silently
                  Justin Bertram Master

                  It appears to me that the JBOSS_SERVER.log you attached is missing some information (i.e. where the "..." entries appear).  Can you confirm this?  If so, please attach the full server log?

                  • 6. Re: MDB reconnect fails silently
                    Michael Woldrich Newbie

                    Sorry, I cleaned the log. I attach the full log.

                    • 7. Re: MDB reconnect fails silently
                      Michael Woldrich Newbie

                      Hello, could you please tell me if there are any results and if you can reproduce the problem?

                      Do you think it is better to open a issue.

                       

                      Greetings

                      • 8. Re: MDB reconnect fails silently
                        Justin Bertram Master

                        I haven't attempted to reproduce the issue.  Feel free to provide an automated (as much as possible) test-case that reproduces the issue.

                         

                        In the mean-time, can you compress and attach your deploy/hornetq directory? I noticed in your log that you're deploying a file named mpa-hornetq-jms.xml and I'm curious what it contains.  My hunch is that it contains the destination which the MDB is looking for and it isn't being deployed quickly enough because that file is scanned and deployed after the HornetQ broker itself is deployed.

                        • 9. Re: MDB reconnect fails silently
                          Michael Woldrich Newbie

                          Hello.

                          Yes in the mpa-hornetq-jms.xml we deploy our queues. But in hornetq-jms.xml is the DLQ defined and in logfile I see that this queue is deployed at the same time as our queues.

                           

                          2012-10-05 14:24:38,976 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.DLQ

                          2012-10-05 14:24:39,077 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.ExpiryQueue

                          2012-10-05 14:24:39,098 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.RBS_Dokument_Error

                          2012-10-05 14:24:39,109 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.gfTestQueueXa

                          2012-10-05 14:24:39,119 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.testQueue

                          2012-10-05 14:24:39,130 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.RBS_Dokument

                          2012-10-05 14:24:39,139 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.CFTProcessingMDB

                          2012-10-05 14:24:39,150 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.Rules_Test

                          2012-10-05 14:24:39,160 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.Rules_Test2

                          2012-10-05 14:24:39,169 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.gfTestQueue

                          2012-10-05 14:24:39,179 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.topic.update

                          2012-10-05 14:24:39,208 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.RBS_Test2_Error

                          2012-10-05 14:24:39,217 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.RBS_Test

                          2012-10-05 14:24:39,228 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.topic.bms

                          2012-10-05 14:24:39,236 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.topic.impExpTopic

                          2012-10-05 14:24:39,244 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.Rules_Test_Error

                          2012-10-05 14:24:39,253 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.RBS_Test_Error

                          2012-10-05 14:24:39,262 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.topic.gfTestTopicXa

                          2012-10-05 14:24:39,270 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.queue.RBS_Test2

                          2012-10-05 14:24:39,279 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.topic.pool

                          2012-10-05 14:24:39,287 INFO  [Thread-10] org.hornetq.core.server.impl.HornetQServerImpl : trying to deploy queue jms.topic.gfTestTopic