8 Replies Latest reply on Aug 4, 2010 2:55 AM by andreas_back

    JMSRedelivered after client crash

    andreas_back

      Hello HornetQ-Team!

       

      It is very inspiring to test HornetQ and thank you for producing such a fast messaging server.

       

      While migrating from JBoss MQ we have seen some point according to message redelivery and client crash in JBoss Messaging, see https://community.jboss.org/thread/151962.

       

      This has led to a small test program where we investigated the reaction of HornetQ in this case. The following observation has been made.

       

      If the client crashes (for example shutdown of the JVM within the IDE) after the message has been received by (MessageConsumer c).receive(1000) but before it is acknowledged by the client then the following happens.

       

      1.     The message is not visible to the QueueBrowser until the server detects the clients crash and releases the ressources.

      2.     If the message is visible to the QueueBrowser again the QueueBrowser reports JMSXDeliveryCount = 0 and getJMSRedelivered() =false.

      3.     If after this the client connects to the server again the message then is redeliverd to the client with JMSXDeliveryCount = 1 and getJMSRedelivered() = false.

       

      But the content of the flags JMSXDeliveryCount and JMSRedelivered that are observed in step 3 do not differ from a first time delivery of a message.

       

      With JBoss Messaging 1.4.3 and 1.4.6 we have seen a similar behaviour as in HornetQ in the case of a client crash.

       

      JBoss MQ sets JMS_JBOSS_REDELIVERY_COUNT = 0 for fresh messages and JMS_JBOSS_REDELIVERY_COUNT = 1 for messages that are re-delivered after a client crash.

       

      Questions:

       

      *     Is it a bug or a feature that HornetQ 2.0.0.GA does react in this way?

      *     Are there parameters to influence the reations of HornetQ to behave an a way that is analog to JBoss MQ?

       

      Greetings

       

      Andreas

        • 1. Re: JMSRedelivered after client crash
          clebert.suconic

          Look at the TTL chaper on the documentation:

           

           

          http://hornetq.sourceforge.net/docs/hornetq-2.0.0.GA/user-manual/en/html_single/index.html#connection-ttl

           

           

          You can configure the TTL accordingly to your needs.

           

           

          As explained on the docs.. the client has a chance to reconnect. Say you have a temporary connection failure. You could tweak this config if you need.

          • 3. Re: JMSRedelivered after client crash
            andreas_back

            Hello Tim and Clebert,

             

            thank you for spending your valuable time and your quick response!

             

            1.     Yes, Tim, the parameter

             

            persist-delivery-count-before-delivery

             

            has been set to true, but this does not influence the test result in step 3 above. The

             

            • fresh and
            • redelivered message


            have both the parameters


            • JMSXDeliveryCount = 1
            • getJMSRedelivered() = false.

             

            2.     The tests have shown that the connection TTL does influence the time until the ressources are released by the server (as described in the user manual) and therefore unitl the message is visible to the QueueBrowser again.

             

            Thank you Clebert for pointing me to this part of the user manual!

             

            This parameter is important for us because the messages in the queue initiate the processing of catalogues on a cluster of service units that can last for hours.

             

            But the present focus while migrating the application that runs for about five years is to prevent the reprocessing of jobs that have caused a system error on the service unit and crashed the JVM. In the old system we move those messages that would be redelivered into the  DLQ by a MDB.

             

            We have implemented a reconnection feature in the client in the case of lost connections (and use a message listener in the productive client application). But crashed JVMs must be restarted by the system administrator.

             

             

            3.     Meanwhile I  have had a closer look to the JMS 1.1 specification. Under "4.10 Reliability" it does state on page 72:

             

            "Unacknowledged messages redelivered due to system failure must have the JMSRedelivered message header field set by the JMS provider."

             

            On the first view there may be a discrepancy between the test result noted in this email under point 1 and this requirement in the JMS specification.

             

            Greetings

             

            Andreas

            • 4. Re: JMSRedelivered after client crash
              timfox

              I'm a little confused about this thread.

               

              Can you provide a simple test program which replicates the issues, that does not use QueueBrowser (or does this only occur with QueueBrowser)?

               

              If you set persist-delivery-count-before-delivery then if the server crashes after delivery but before ack the delivery count should be higher on the next delivery attempt after recovery.

              • 5. Re: JMSRedelivered after client crash
                andreas_back

                Hello Tim,

                 

                thank you for your interest in this point! The simple test class is added to this posting.

                 

                As mentioned in the starting message of this thread the test covers the case that the - client - crashes.

                 

                As far as I can see it the QueueBrowser does not matter. Therefore the test class does not contain it. It just has been used to see when the TTL is over and the message is available on the queue again. (Our application has a graphical utility that browses the queues.)

                 

                Greetings

                 

                Andreas

                • 6. Re: JMSRedelivered after client crash
                  timfox
                  • 7. Re: JMSRedelivered after client crash
                    andreas_back

                    Hello Tim,

                     

                    thank you for your really fast reaction to this point!

                     

                    I hope that HornetQ 2.1.0.GA will come soon and within the time frame of our projekt, so that our project management can be convinced to migrate to it!

                     

                    Greetings

                     

                    Andreas

                    • 8. Re: JMSRedelivered after client crash
                      andreas_back

                      Hello Tim,

                       

                      yes, your fix has been tested in our application using the release 2.1.1.Final.

                       

                      Best regards,

                       

                      Andreas