10 Replies Latest reply on Apr 20, 2011 4:52 AM by Tomasz Kwiatkowski

    message redelivery after consumer failure?

    Steven Marcus Newbie

      We're trying out hornetq 2.1.2 with Spring's DefaultMessageListenerContainer to deliver messages via a queue to slow consumers using sessionTransacted acknowledgement.

       

      It appears that a consumer that dies/is killed while processing a message leaves that message "in limbo".

      That is, the message is not redelivered -- even after the (default) connectionTTL has elapsed.

      But, a manual restart of the hornetq server does cause the limbo message to be redelivered to a consumer.

       

      The DMLC is using its own ConnectionFactory explicitly configured with client-window-size = 0.

      Not sure if this makes a difference but paging is enabled on the server -- although this behavior can be reproduced with a single message in the queue and a single consumer.

       

      I've done a brief search of the forum and bugs database and seen some related issues -- but nothing that leads me to believe that this is a known issue with 2.1.2.

       

      Any help greatly appreciated!

      Steven M.

        • 1. Re: message redelivery after consumer failure?
          Andy Taylor Master

          can you provide a simple test case, i.e. not using spring, that demonstrates the issue.

          • 2. message redelivery after consumer failure?
            Thomas Frers Newbie

            Hi,

             

            is there a solution for this issue. We' ve got the same problem. The message is not redelivered when we are using a DefaultMessageListenerContainer.

             

            Thanks,

            Thomas

            • 3. message redelivery after consumer failure?
              Andy Taylor Master

              as my last post, if you can provide a test case that doesn't use spring we can take a look, typically these are spring issues not messaging.

              • 4. message redelivery after consumer failure?
                Thomas Frers Newbie

                Hi there!

                 

                I think I've got a solution on this.

                 

                You have to set an ErrorHandler to the DefaulMessageListenerContainer like that:

                 

                 

                listenerContainer.setErrorHandler(new ErrorHandler()

                    {

                      @Override

                      public void handleError(Throwable t)

                      {

                         throw new RuntimeException(t);

                      }

                    });

                 

                listenerContainer is an instance of DefaulMessageListenerContainer.

                 

                So the listenerContainer stops and reinitialize.

                 

                Greetings,

                Thomas

                 

                 

                 

                 

                • 5. message redelivery after consumer failure?
                  Clebert Suconic Master

                  Throwing an exception inside an Exception handler sounds like a hack to me.

                   

                  Anyway... As you point out, that's making the listenerContainer to re-initialize, what makes it a SpringIssue.

                   

                   

                  If I was a spring developer now I would probably tell you to look at a way to proper re-initialize it. a method.. close/restart... anything.

                   

                   

                  But if you're happy with the hack... then it's up to you

                  • 6. Re: message redelivery after consumer failure?
                    Tomasz Kwiatkowski Newbie

                    Hi,

                     

                    I have a little bit similar scenario (without using spring):

                     

                    - client connects to HornetQ server, sends message to queue A, and is waiting for a reply in queue B

                    - client is killed without closing connection

                    - some other application listens on queue A, produces response and injects into queue B

                    - another client application connects, sends message to queue A and is listening to queue B

                    - nothing comes to the second application (HornetQ probably tries to send to the first client)

                    - HornetQ detects that application died and clears up the resources

                    - still nothing comes to the second client

                    - second client is closed

                    - client application is restarted

                    - two pending messages comes immediatly to the application from queue B

                     

                    Should I post something from code or configuration? Is this normal situation or should messages come to the second client right after server-side cleanup?

                     

                    I'm currently on

                    Hi,

                     

                    I have a little bit similar scenario (without using spring):

                     

                    - client connects to HornetQ server, sends message to queue A, and is waiting for a reply in queue B

                    - client is killed without closing connection

                    - some other application listens on queue A, produces response and injects into queue B

                    - another client application connects, sends message to queue A and is listening to queue B

                    - nothing comes to the second application (HornetQ probably tries to send to the first client)

                    - HornetQ detects that application died and clears up the resources

                    - still nothing comes to the second client

                    - second client is closed

                    - client application is restarted

                    - two pending messages comes immediatly to the application from queue B

                     

                    ShouldI post something from code or configuration? Is this normal situationor should messages come to the second client right after server-sidecleanup?

                     

                    I'm currently on 2.1.2.Final

                    • 7. Re: message redelivery after consumer failure?
                      Clebert Suconic Master

                      The message should be redelivered as soon as the server loses pings more more than TTL. At that point the message is redelivered to live clients.

                       

                       

                      Notice that temporary queues are deleted right away in case of client failure. (The temporary queue is deleted when the connection dies)

                      • 8. Re: message redelivery after consumer failure?
                        Tomasz Kwiatkowski Newbie

                        Thanks for response.

                         

                        The problem is that my last client is live for sure, the only problem is that it became live between sending the message to the dead client and cleaning after TTL expiration. If this last client from the scenario is started a little bit later (but in exactly the same way and it is in the same state) - after throwing out timeouted client, everything works fine and message is redelivered correctly.

                         

                        To sum up:

                         

                        - client 1 is killed

                        - server is sending message A to client 1

                        - client 2 comes into the game

                        - server recognize client 1 as dead

                        - client 2 doesn't receive message A as I would expect (it's live and running)

                        - client 3 is started and it immediatly receives the message A

                         

                        Is my problem more clear now?

                        • 9. Re: message redelivery after consumer failure?
                          Clebert Suconic Master

                          You should first try the latest version.

                           

                          If it is still happening, you should probably create a test showing the issue, and we can take a look.

                           

                           

                           

                          I'm not aware of any other issues like this on the latest version.

                          • 10. Re: message redelivery after consumer failure?
                            Tomasz Kwiatkowski Newbie

                            You're right, I should try the latest stable first. I'll do this excersise and let you know what happened, thanks.