12 Replies Latest reply on Jul 14, 2006 12:46 PM by Adrian Brock

    Session recovery on rollback

    Tim Fox Master

      When a jms session is rolled back via the Session.rollback() method, any messages due to be sent and any acks due to be made in the session are thrown away.

      Also any messages delivered in that session are supposed to be immediately redelivered, in the same way as what happens when you call session.recover().

      This is all pretty clear from the spec.

      I wanted to clarify the recovery behaviour when the session is enlisted in an XA transaction and rollback is called on the sessions XAResource.

      Clearly the messages and acks need to be discarded in the same way, but I'm not sure if we are required to replay the delivered messages to the sessions consumers as we are when rolling back using session.recover().

        • 1. Re: Session recovery on rollback
          Adrian Brock Master

          That's twice you've stated a non-requrement in the spec.

          The only time the spec mentions anything related to the *timing* of
          redelivery is when dealing with runtime exceptions thrown
          by message listeners in AUTO/DUPS_OK acknowledge mode:

          JMS1.1 - 4.5.2
          "
          The result of a listener throwing a RuntimeException depends on the session?s
          acknowledgment mode.
          ? AUTO_ACKNOWLEDGE or DUPS_OK_ACKNOWLEDGE - the message
          will be immediately redelivered. The number of times a JMS provider will
          redeliver the same message before giving up is provider-dependent. The
          JMSRedelivered message header field will be set for a message redelivered
          under these circumstances.
          "

          Since this part of the spec defines something that is non-portable
          behaviour "provider-dependent", I don't see why you are trying to
          expand this to rest of the JMS spec?

          In fact, if your interpretation was true, JBossMQ (and other jms
          providers) would not be allowed to have the "redelivery delay"
          feature that stops cpu loops in the event of a broken listener/subscriber.

          • 2. Re: Session recovery on rollback
            Adrian Brock Master

            On the XAResource question, the timing is completely out of your control.

            It could be an infinite delay if the TM's transaction log is not recovered
            after a prepared JTA transaction and then a crash.

            All you can do is put the messages back in the queue/subscription
            when the TM does decide to rollback.
            This could be "long after" the consumer, session or connection
            has disappeared.

            • 3. Re: Session recovery on rollback
              Adrian Brock Master

               

              "adrian@jboss.org" wrote:

              It could be an infinite delay if the TM's transaction log is not recovered
              after a prepared JTA transaction and then a crash.


              I assume no heuristics.

              • 4. Re: Session recovery on rollback
                Tim Fox Master

                Point taken, the timing isn't important, but my understanding is the order is.

                So if I receive messages A, B, C then rollback I should receive A, B, C.

                The only exception to this if some of those messages have since expired or higher priority messages have arrived, in all other cases the order must be preserved. Is this correct?

                In JBossMQ SpySession there is this comment:

                "// Restart the delivery sequence including all unacknowledged messages
                // that had
                // been previously delivered. Redelivered messages do not have to be
                // delivered
                // in exactly their original delivery order."

                Which seems to conflict with what the spec says.. although I guess I'm misinterpreting it wrong, although not sure where :( I would never make a good lawyer...

                Perhaps I am confusing the order of receipt of the messages with the order of delivery of the messages.

                E.g.

                If I received A, B, C then rolled back and then received

                A, D, B, C, E

                then the delivery order is still preserved (A, B, C) it's just they're interleaved with D and E.
                ??

                JMS1.1 4.4.11:

                "In effect, the session?s series of delivered messages
                is reset to the point after its last acknowledged message. The messages it now
                delivers may be different from those that were originally delivered due to
                message expiration and the arrival of higher-priority messages."

                • 5. Re: Session recovery on rollback
                  Adrian Brock Master

                  You have it correct. Order is different from timing.

                  The comment in the SpySession is just saying the same thing as the spec.

                  The implementation in SpySession is overly complicated because there
                  is no "session thread" which means it has to stop delivery on the
                  whole connection to guarantee no new deliveries during the recovery.

                  • 6. Re: Session recovery on rollback
                    Tim Fox Master

                     

                    "adrian@jboss.org" wrote:

                    The implementation in SpySession is overly complicated because there
                    is no "session thread" which means it has to stop delivery on the
                    whole connection to guarantee no new deliveries during the recovery.


                    Make sense. So *somehow* it's necessary to prevent any new messages arriving while recover is being executed.

                    In the same way, It's my understanding that we would need to also prevent any new messages arriving while rollback is being executed. (Since the spec says rollback cause session recovery)

                    Looking at the SpySession code I don't see the connection being stopped while rollback is happening. So it looks like messages could creep in which would be non spec complient.

                    (BTW I'm not criticising the spy code, I'm just trying to make sure we're doing it correctly in messaging :) )

                    • 7. Re: Session recovery on rollback
                      Tim Fox Master

                      If it was acceptable for ordering to be lost after rollback that makes my life a lot easier.

                      This JIRA task seems to imply that it is not a requirement of the spec.

                      http://jira.jboss.com/jira/browse/JBAS-2444

                      Although I don't really understand how it is not a requirement since the spec says that session recovery occurs on rollback, and session recovery maintains the order (if prioriry and expiration are taken out of the equation).

                      • 8. Re: Session recovery on rollback
                        Adrian Brock Master

                        You are still confusing different things.
                        The JMS Session is not the same as the transaction.

                        In fact, with transaction interleaving it is perfectly legal
                        (and more efficient) to do the following which JBossMQ actually supports:

                        1) Enlist session in tx1
                        2) receive message 1 (this is in the session, but actually belongs to tx1)
                        3) delist session
                        4) Enlist session in tx2
                        5) receive message 2 (into tx2)
                        6) commit tx1 (this does NOT commit the session, it commits the transaction, i.e. message 1)

                        Again, there is a test for this in the JBossMQ testsuite.

                        Similarly, these could be different sessions on different threads
                        receiving from the same queue.

                        The idea of the JIRA task is that step 5 would not receive message 2
                        until step 6 either commits or rollbacks message 1. If it rolls back
                        then step 5 gets message 1.

                        This maintains a GLOBAL ordering across transactions/threads/sessions,
                        something that is not a spec requirement, but some people have
                        asked for.
                        I wouldn't recomend somebody uses this, unless they aren't worried
                        about throughput. :-)

                        • 9. Re: Session recovery on rollback
                          Adrian Brock Master

                          Incidently, if that JIRA task was done and being used
                          and the above example was the same session on the same thread,
                          it would DEADLOCK. Step 5 would never complete because step 6
                          can't happen.

                          • 10. Re: Session recovery on rollback
                            Tim Fox Master

                            What you're saying is that if you put the message back on the queue (nack it) on rollback then you can't guarantee ordering, since the session could do work in different transactions that could rollback in a different order.

                            However if you're not using XA then it seems to me that order can be maintained, even if you have competing consumers on the same queue, since on rollback you don't need to NACK the message back to the queue (where it could get picked up by a competing consumer), instead you can just recover it locally in the session, i.e. just resubmit it to the onMessage method, in the same way you do with session.recover.

                            • 11. Re: Session recovery on rollback
                              Tim Fox Master

                              One of the reasons I'm going on about all this, is that messaging now (or very shortly) supports prefetching of messages (which incidentally gives us a massive boost in throughput :) ), so when I rollback a session, if I have to nack the messages delivered, I also have to nack all the prefetched ones on the client back to the server - which i want to avoid doing If I can avoid it, otherwise the order is going to be screwed when the nacked ones are redelivered (the prefetched ones will have jumped ahead).

                              If I can deal with rollback locally (I.e. not nack the messages) I can just re-insert the delivered messages at the front of the prefetched messages on the client side and everything is much easier (and more performant - although performance for rollback is probably not an important issue)

                              • 12. Re: Session recovery on rollback
                                Adrian Brock Master

                                Nearly correct, except that you still can't guarantee the ordering if the
                                messages are truly nacked, e.g. consumer or session.close();

                                e.g. competing consumers on the same queue messages (m1, m2)
                                consumer1: receive m1
                                consumer2: receive m2
                                Race
                                {
                                consumer1.close();
                                consumer2.rollback();
                                }
                                what does consumer2.receive() get?

                                If you can avoid nacking the message back to the server
                                then yes, you avoid *some* race conditions where the messages
                                get "out of order".

                                Since those race conditions exist, and hence the ordering
                                ambiguities exist, it isn't necessarily
                                worthwhile trying to achieve the impossible.
                                Don't spend 90% of your time on 10% of the problem. :-)