1 2 Previous Next 18 Replies Latest reply on Feb 25, 2009 3:02 AM by timfox

    DeliveryCount & ACKs

    clebert.suconic

      I have been working with https://jira.jboss.org/jira/browse/JBMESSAGING-1294, and this is the current scenario:

      - The MessageReference.deliveryCount is only updated after ACKs.

      Say, if you:
      - received a message
      - called message.acknowledge
      - called rollback

      When you receive that same message again, the deliveryCounter will be incremented, as the ACK will have it incremented:

      Class: ServerConsumerImpl
      {
      ...
      Method acknowledge(final boolean autoCommitAcks, final Transaction tx, final long messageID)
      {
      ...
       if (autoCommitAcks)
       {
       ref.getQueue().acknowledge(ref);
       }
       else
       {
       ref.getQueue().acknowledge(tx, ref);
      
       // Del count is not actually updated in storage unless it's
       // cancelled
       ref.incrementDeliveryCount();
       }
      }
      



      If you are using AUTO_ACK (In JMS Terms that means non-transacted), that means you won't ever need to increment the DeliveryCount, as the message will be committed as soon as you called ack.

      However if you are using MessageListener, that's not true. If you receive a message and an exception was thrown, the counter should be incremented also. But the ClientConsumer can't call ACK, as that would remove it from the Queue.

      So, that means that Acknowledge requires a different semantic when an exception is caught on JMSMessageListenerWrapper.


      One way to fix this, would be to change the ACK call signature on ClientSessionInternal:

       void acknowledge(long consumerID, long messageID) throws MessagingException; // Maybe I would leave this
      
       void acknowledge(long consumerID, long messageID, boolean autoACK) throws MessagingException;
      




      And have the JMSMessageListenerWrapper doing:

       catch (RuntimeException e)
       {
       if (!transactedOrClientAck)
       {
       try
       {
      
       consumer.acknowledge(jbm.getCoreMessage(), false);
      
       session.getCoreSession().rollback();
      
       session.setRecoverCalled(true);
       }
       catch (Exception e2)
       {
       log.error("Failed to recover session", e2);
       }
       }
       }
      




      With that, the AutoACK property would aways be sent on SessionAcknowledgeMessage, leaving the AutoACK to be controlled from the client.

      (I could extend the packet if the boolean is considered a problem, & if this solution is acceptable).


      The only problem I found on this solution, is requiring a type-cast to ClientConsumerInternal on JBossMessageConsumer.


      Any thoughts?



        • 1. Re: DeliveryCount & ACKs
          clebert.suconic

          With this change:

          catch (RuntimeException e)
           {
           if (!transactedOrClientAck)
           {
           try
           {
          
           consumer.acknowledge(jbm.getCoreMessage(), false);
          
           session.getCoreSession().rollback();
          
           session.setRecoverCalled(true);
           }
           catch (Exception e2)
           {
           log.error("Failed to recover session", e2);
           }
           }
           }
          


          the message would be sent to the ACK list on the transaction before the rollback, so the counter would be incremented.

          • 2. Re: DeliveryCount & ACKs
            timfox

            I think the real problem is, we should be incrementing the delivery count *before* we send the message to the client.

            • 3. Re: DeliveryCount & ACKs
              timfox

              Also see https://jira.jboss.org/jira/browse/JBMESSAGING-1339.

              We should come up with a consistent solution

              • 4. Re: DeliveryCount & ACKs
                clebert.suconic

                 

                "timfox" wrote:
                I think the real problem is, we should be incrementing the delivery count *before* we send the message to the client.


                We can't do that, as we need to distinguish messages sent to the buffer, from messages consumed on the client.

                • 5. Re: DeliveryCount & ACKs
                  timfox

                  Not sure I agree.

                  In JMS JMSRedelivered=true means the message was *probably* delivered before.

                  But any message that has been delivered before *must* have JMSDelivered set to true.

                  You can only do that by incrementing it on the server side before delivery to the client.

                  Really we should also persist that info in storage too before delivery to be really JMS compliant. We can have a flag to do that and default it to false.

                  • 6. Re: DeliveryCount & ACKs
                    clebert.suconic

                     

                    "timfox" wrote:
                    Also see https://jira.jboss.org/jira/browse/JBMESSAGING-1339.

                    We should come up with a consistent solution


                    What about looking for Updates Rollback Transactions during journal.reload?

                    Maybe you could have a case where a file could be reclaimed with the UpdateRecord... but that seems a pretty low probability, since the Update will come after the Add, and the AddRecord would still be valid during reload.

                    • 7. Re: DeliveryCount & ACKs
                      clebert.suconic


                      What about looking for Updates Rollback Transactions during journal.reload?


                      I meant, updates on Rolled back transaction.. but I wold do that also on reading incomplete transaction. (thinking of a crash).



                      • 8. Re: DeliveryCount & ACKs
                        timfox

                        Sorry, I didn't understand your last two posts on this thread, and what relevance they had.

                        • 9. Re: DeliveryCount & ACKs
                          clebert.suconic

                          Just to complete the thread,

                          What I was suggesting was to keep incrementing deliveries only after ACK, and recover the delivery counter from the journal, by inspecting rolled back data.


                          But as we talked on IRC, I only created a flag, and if that flag is set we would update the deliveryCount after sending.

                          I have created a flag StrictUpdateDelivery on the Configuration.

                          (If anyone has a better name....).

                          I thought about calling it StrictJMS, but that wouldn't be a good name

                          Also, another option I considered was to place the attribute on the SessionFactory (or JMSConnectionFactory).

                          If Tim or anyone think it's better rename or to move the attribute anywhere, it should be an easy change.

                          • 10. Re: DeliveryCount & ACKs
                            timfox

                             

                            "clebert.suconic@jboss.com" wrote:
                            Just to complete the thread,

                            What I was suggesting was to keep incrementing deliveries only after ACK, and recover the delivery counter from the journal, by inspecting rolled back data.


                            But as we talked on IRC, I only created a flag, and if that flag is set we would update the deliveryCount after sending.


                            DeliveryCount needs to be updated *before* delivery, not after.

                            If flag persistDeliveryCountBeforeDelivery = true then you also need to persist the update in storage *before* delivery.

                            If flag persistDeliveryCountBeforeDelivery = false then you can defer updating delivery count in storage until the reference is cancelled.



                            • 11. Re: DeliveryCount & ACKs
                              timfox

                              Also what's with changing the tests semantics?

                              JBMESSAGING-1339 and JBMESSAGING-1294 - Fixing tests after behaviour changes on DeliveryCounters
                              


                              The client behaviour shouldn't change....

                              Also you need core integration tests that test delivery count and starting and stopping the server. The old JMS tests will be removed soon so please don't add any more!

                              • 12. Re: DeliveryCount & ACKs
                                clebert.suconic

                                There was jms-tests validating the delivery-counter. We are updating the message as soon as we deliver, and those tests were receiving messages several more times without ACKing them.

                                We used to update delivery counters only on ACK. Now that we update them before delivery, I had to change thos JMStests.


                                Also you need core integration tests that test delivery count and starting and stopping the server


                                I will probably come up with more tests, but so far I have this:

                                http://anonsvn.jboss.org/repos/messaging/trunk/tests/src/org/jboss/messaging/tests/integration/consumer/RedeliveryConsumerTest.java

                                • 13. Re: DeliveryCount & ACKs
                                  timfox

                                  The Jms tests should not change. If the behaviour has changed to the client, then you haven't implemented it properly

                                  • 14. Re: DeliveryCount & ACKs
                                    clebert.suconic

                                    Few posts ago I said:


                                    We can't do that, as we need to distinguish messages sent to the buffer, from messages consumed on the client.


                                    And you said that based on the Spec, it should be ok to update the counter as soon as it was sent to the buffer on the client, Those tests are validating that behaviour. (A message sent to the client, but not consumed).

                                    1 2 Previous Next