13 Replies Latest reply on Mar 27, 2006 10:44 AM by Tim Fox

    Order of message acknowledgments in a session / xasession

    Tim Fox Master

      Just wanted to clarify something.

      For a non XA Session, messages will always be acked in the order they were received, this is either at tx commit time, message delivery time (auto_ack), message.acknowledge time, or lazily (with dups).

      For an XA session, it can be enrolled in JTA tx1, receive some messages, then be enrolled in another JTA tx2. (as long as the start and end calls don't overlap this is fine).

      Some time later tx2 can be comitted before tx1, resulting in later messages being acked before earlier ones.

      Is this correct reasoning?

      The reason I am asking is that if we can assume that message acknowledgement for a queue or subscription occurs in the order of delivery, then we can make some optimisations, but I don't think this is always true...

        • 1. Re: Order of message acknowledgments in a session / xasessio
          Adrian Brock Master

           

          "timfox" wrote:

          The reason I am asking is that if we can assume that message acknowledgement for a queue or subscription occurs in the order of delivery, then we can make some optimisations, but I don't think this is always true...


          It is true for topics. It isn't true for contested queues.

          • 2. Re: Order of message acknowledgments in a session / xasessio
            Tim Fox Master

            Even for a non contested (non contested means only one consumer??) queue - I'm not sure what's preventing you enrolling the session in two different JTA txs, one after another, and then committing the txs in reverse order of the order in which the sessions were enrolled:

            e.g.

            
            xaresource.start(tx1)
            
            //receive some messages
            
            xaresource.end(tx1)
            
            xaresource.start(tx2)
            
            //receive some messages
            
            xaresource.end(tx2)
            
            
            //now commit in different order
            
            xaresource.commit(tx2) // second lot of messages acked
            
            xaresource.commit(tx1) // first lot of messages acked
            
            


            My understanding is this will cause messages to be acked in different order to delivery order?

            I must be missing something...

            • 3. Re: Order of message acknowledgments in a session / xasessio
              Adrian Brock Master

              Potentially yes. An MDB does exactly what you describe.

              It's debatable whether you would describe this as uncontested
              since you have concurrent sessions/transactions.

              • 4. Re: Order of message acknowledgments in a session / xasessio
                Tim Fox Master

                Ok, so i the general case i can't make any assumptions as to the order of acks.

                The same reason must apply to cancels too, since they would occur on tx rollback (cancel= put message back on beginning of queue).

                This implies that, in general I cannot make any assumptions there is any global order of messages in queues / subs, since messages can be cancelled i a different order resulting in them being put back in a different position on the queue to where they were before.

                The reason I was thinking about this, is in the paging case (lazy loading queue), when loading the next x references from the queue, if I can say something like;

                
                select blah from message_reference where ordering between a and b
                
                


                This is going to be a lot quicker than selecting all the refs in the queue then just using the first x, especially when there are million messages in the queue, since for some dbs/drivers they will get the entire resultset before sending back.

                I tested this last night with mysql. I successfully sent 1000000 2K messages to a queue - this works fine.

                However when receiving the loads are quite slow when there are a lot of messages for the above reason.

                I guess we can limit this by using db specific stuff which allows you to only select the first x messages - but I'd like to avoid db specific stuff.

                Any suggestions?


                • 5. Re: Order of message acknowledgments in a session / xasessio
                  Adrian Brock Master

                  Speak to the hibernate guys.

                  • 6. Re: Order of message acknowledgments in a session / xasessio
                    Adrian Brock Master

                    The usual trick is to remember the "last key" and unions.
                    But I don't know how portable unions, both semantically
                    and in terms of performance?

                    e.g. if you know the last message in the queue was 27
                    you do

                    select blah from message_reference where priority = 1 and msgid > 27
                    union
                    select blah from message_reference where priority = 2 and msgid > 27
                    etc.
                    


                    This is also only going to scale well if the result set
                    doesn't try to load the entire database into memory
                    (like mysql does),



                    • 7. Re: Order of message acknowledgments in a session / xasessio
                      Adrian Brock Master

                      I wrote the union thing that way (missed the order by :-)
                      because the alternative to unions is multiple queries.

                      • 8. Re: Order of message acknowledgments in a session / xasessio
                        Aaron Walker Newbie

                        In general terms even if your not using a full blown Hibernate implementation and you want to avoid db specific stuff SQL you can use the Hibernate Dialect classes to generate the appropriate SQL.

                        • 9. Re: Order of message acknowledgments in a session / xasessio
                          Tim Fox Master

                          There is an interesting implication to this message cancellation stuff that I do not fully understand.

                          If it is possible for messages to change their position in the queue due to be cancelled due to two or more JTA transactions rolling back in a different order, then how can we satisfy the JMS ordering guarantee whereby messages from a session should be received in the same order they were sent.

                          Doesn't make sense to me.

                          • 10. Re: Order of message acknowledgments in a session / xasessio
                            Adrian Brock Master

                            You don't need to support ordering when a message is redelivered.

                            Doing so would be very unperformant:
                            http://jira.jboss.com/jira/browse/JBAS-2444

                            • 11. Re: Order of message acknowledgments in a session / xasessio
                              Tim Fox Master

                              Not having to support ordering after failure would make my life easier

                              I looked through the spec trying to find where it says the message ordering guarantee doesn't apply in redelivery but couldn't find it.

                              The nearest I can find is:


                              4.4.11
                              A session?s recover method is used to stop a session and restart it with its first
                              unacknowledged message. In effect, the session?s series of delivered messages
                              is reset to the point after its last acknowledged message. The messages it now
                              delivers may be different from those that were originally delivered due to
                              message expiration and the arrival of higher-priority messages.


                              But this only mentions expiration or higher priority messages which is not what is happening in our case.

                              Is this mentioned somewhere else?



                              • 12. Re: Order of message acknowledgments in a session / xasessio
                                Adrian Brock Master

                                Think about it. How you can guarantee the delivery order
                                when there are multiple sessions/transactions nacking messages.
                                It is entirely dependent on the timing of the clients' operations.

                                You could even have

                                client 1: nack message1
                                client 2: nack message2
                                client 2: receive gets message1
                                client 1: receive gets message2

                                where the messages delivered aren't even preserved let alone
                                the ordering. :-)

                                This does not apply to your quote, which is about client acknowledge.
                                The messages are "nacked" into the session (waiting for a recover() or
                                a subsequent ack), not the queue for client acknowledge.
                                So nobody else can see them unless the session is closed.

                                • 13. Re: Order of message acknowledgments in a session / xasessio
                                  Tim Fox Master

                                  Agreed, it would be hard to guarantee ordering without imposing some kind of serialization of access to the channel from sessions/transactions.

                                  So I guess then we can imply it is ok then to relax the guarantee in this situation since it might have an "unreasonable" effect on performance in order to maintain it.