Session recovery on rollback

[timfox]

When a jms session is rolled back via the Session.rollback() method, any messages due to be sent and any acks due to be made in the session are thrown away.

Also any messages delivered in that session are supposed to be immediately redelivered, in the same way as what happens when you call session.recover().

This is all pretty clear from the spec.

I wanted to clarify the recovery behaviour when the session is enlisted in an XA transaction and rollback is called on the sessions XAResource.

Clearly the messages and acks need to be discarded in the same way, but I'm not sure if we are required to replay the delivered messages to the sessions consumers as we are when rolling back using session.recover().

[adrian.brock]

That's twice you've stated a non-requrement in the spec.

The only time the spec mentions anything related to the *timing* of
redelivery is when dealing with runtime exceptions thrown
by message listeners in AUTO/DUPS_OK acknowledge mode:

JMS1.1 - 4.5.2
"
The result of a listener throwing a RuntimeException depends on the session?s
acknowledgment mode.
? AUTO_ACKNOWLEDGE or DUPS_OK_ACKNOWLEDGE - the message
will be immediately redelivered. The number of times a JMS provider will
redeliver the same message before giving up is provider-dependent. The
JMSRedelivered message header field will be set for a message redelivered
under these circumstances.
"

Since this part of the spec defines something that is non-portable
behaviour "provider-dependent", I don't see why you are trying to
expand this to rest of the JMS spec?

In fact, if your interpretation was true, JBossMQ (and other jms
providers) would not be allowed to have the "redelivery delay"
feature that stops cpu loops in the event of a broken listener/subscriber.

[adrian.brock]

On the XAResource question, the timing is completely out of your control.

It could be an infinite delay if the TM's transaction log is not recovered
after a prepared JTA transaction and then a crash.

All you can do is put the messages back in the queue/subscription
when the TM does decide to rollback.
This could be "long after" the consumer, session or connection
has disappeared.

[adrian.brock]

It could be an infinite delay if the TM's transaction log is not recovered
after a prepared JTA transaction and then a crash.

I assume no heuristics.

[timfox]

Point taken, the timing isn't important, but my understanding is the order is.

So if I receive messages A, B, C then rollback I should receive A, B, C.

The only exception to this if some of those messages have since expired or higher priority messages have arrived, in all other cases the order must be preserved. Is this correct?

In JBossMQ SpySession there is this comment:

"// Restart the delivery sequence including all unacknowledged messages
// that had
// been previously delivered. Redelivered messages do not have to be
// delivered
// in exactly their original delivery order."

Which seems to conflict with what the spec says.. although I guess I'm misinterpreting it wrong, although not sure where :( I would never make a good lawyer...

Perhaps I am confusing the order of receipt of the messages with the order of delivery of the messages.

E.g.

If I received A, B, C then rolled back and then received

A, D, B, C, E

then the delivery order is still preserved (A, B, C) it's just they're interleaved with D and E.
??

JMS1.1 4.4.11:

"In effect, the session?s series of delivered messages
is reset to the point after its last acknowledged message. The messages it now
delivers may be different from those that were originally delivered due to
message expiration and the arrival of higher-priority messages."

[adrian.brock]

You have it correct. Order is different from timing.

The comment in the SpySession is just saying the same thing as the spec.

The implementation in SpySession is overly complicated because there
is no "session thread" which means it has to stop delivery on the
whole connection to guarantee no new deliveries during the recovery.

[timfox]

The implementation in SpySession is overly complicated because there
is no "session thread" which means it has to stop delivery on the
whole connection to guarantee no new deliveries during the recovery.

Make sense. So *somehow* it's necessary to prevent any new messages arriving while recover is being executed.

In the same way, It's my understanding that we would need to also prevent any new messages arriving while rollback is being executed. (Since the spec says rollback cause session recovery)

Looking at the SpySession code I don't see the connection being stopped while rollback is happening. So it looks like messages could creep in which would be non spec complient.

(BTW I'm not criticising the spy code, I'm just trying to make sure we're doing it correctly in messaging :) )

[timfox]

If it was acceptable for ordering to be lost after rollback that makes my life a lot easier.

This JIRA task seems to imply that it is not a requirement of the spec.

http://jira.jboss.com/jira/browse/JBAS-2444

Although I don't really understand how it is not a requirement since the spec says that session recovery occurs on rollback, and session recovery maintains the order (if prioriry and expiration are taken out of the equation).

[adrian.brock]

You are still confusing different things.
The JMS Session is not the same as the transaction.

In fact, with transaction interleaving it is perfectly legal
(and more efficient) to do the following which JBossMQ actually supports:

1) Enlist session in tx1
2) receive message 1 (this is in the session, but actually belongs to tx1)
3) delist session
4) Enlist session in tx2
5) receive message 2 (into tx2)
6) commit tx1 (this does NOT commit the session, it commits the transaction, i.e. message 1)

Again, there is a test for this in the JBossMQ testsuite.

Similarly, these could be different sessions on different threads
receiving from the same queue.

The idea of the JIRA task is that step 5 would not receive message 2
until step 6 either commits or rollbacks message 1. If it rolls back
then step 5 gets message 1.

This maintains a GLOBAL ordering across transactions/threads/sessions,
something that is not a spec requirement, but some people have
asked for.
I wouldn't recomend somebody uses this, unless they aren't worried
about throughput. :-)

[adrian.brock]

Incidently, if that JIRA task was done and being used
and the above example was the same session on the same thread,
it would DEADLOCK. Step 5 would never complete because step 6
can't happen.

[timfox]

What you're saying is that if you put the message back on the queue (nack it) on rollback then you can't guarantee ordering, since the session could do work in different transactions that could rollback in a different order.

However if you're not using XA then it seems to me that order can be maintained, even if you have competing consumers on the same queue, since on rollback you don't need to NACK the message back to the queue (where it could get picked up by a competing consumer), instead you can just recover it locally in the session, i.e. just resubmit it to the onMessage method, in the same way you do with session.recover.

[timfox]

One of the reasons I'm going on about all this, is that messaging now (or very shortly) supports prefetching of messages (which incidentally gives us a massive boost in throughput :) ), so when I rollback a session, if I have to nack the messages delivered, I also have to nack all the prefetched ones on the client back to the server - which i want to avoid doing If I can avoid it, otherwise the order is going to be screwed when the nacked ones are redelivered (the prefetched ones will have jumped ahead).

If I can deal with rollback locally (I.e. not nack the messages) I can just re-insert the delivered messages at the front of the prefetched messages on the client side and everything is much easier (and more performant - although performance for rollback is probably not an important issue)

[adrian.brock]

Nearly correct, except that you still can't guarantee the ordering if the
messages are truly nacked, e.g. consumer or session.close();

e.g. competing consumers on the same queue messages (m1, m2)
consumer1: receive m1
consumer2: receive m2
Race
{
consumer1.close();
consumer2.rollback();
}
what does consumer2.receive() get?

If you can avoid nacking the message back to the server
then yes, you avoid *some* race conditions where the messages
get "out of order".

Since those race conditions exist, and hence the ordering
ambiguities exist, it isn't necessarily
worthwhile trying to achieve the impossible.
Don't spend 90% of your time on 10% of the problem. :-)