Order of message acknowledgments in a session / xasession

[timfox]

Just wanted to clarify something.

For a non XA Session, messages will always be acked in the order they were received, this is either at tx commit time, message delivery time (auto_ack), message.acknowledge time, or lazily (with dups).

For an XA session, it can be enrolled in JTA tx1, receive some messages, then be enrolled in another JTA tx2. (as long as the start and end calls don't overlap this is fine).

Some time later tx2 can be comitted before tx1, resulting in later messages being acked before earlier ones.

Is this correct reasoning?

The reason I am asking is that if we can assume that message acknowledgement for a queue or subscription occurs in the order of delivery, then we can make some optimisations, but I don't think this is always true...

[adrian.brock]

"timfox" wrote:

The reason I am asking is that if we can assume that message acknowledgement for a queue or subscription occurs in the order of delivery, then we can make some optimisations, but I don't think this is always true...

It is true for topics. It isn't true for contested queues.

[timfox]

Even for a non contested (non contested means only one consumer??) queue - I'm not sure what's preventing you enrolling the session in two different JTA txs, one after another, and then committing the txs in reverse order of the order in which the sessions were enrolled:

e.g.

xaresource.start(tx1)

//receive some messages

xaresource.end(tx1)

xaresource.start(tx2)

//receive some messages

xaresource.end(tx2)


//now commit in different order

xaresource.commit(tx2) // second lot of messages acked

xaresource.commit(tx1) // first lot of messages acked

My understanding is this will cause messages to be acked in different order to delivery order?

I must be missing something...

[adrian.brock]

Potentially yes. An MDB does exactly what you describe.

It's debatable whether you would describe this as uncontested
since you have concurrent sessions/transactions.

[timfox]

Ok, so i the general case i can't make any assumptions as to the order of acks.

The same reason must apply to cancels too, since they would occur on tx rollback (cancel= put message back on beginning of queue).

This implies that, in general I cannot make any assumptions there is any global order of messages in queues / subs, since messages can be cancelled i a different order resulting in them being put back in a different position on the queue to where they were before.

The reason I was thinking about this, is in the paging case (lazy loading queue), when loading the next x references from the queue, if I can say something like;

select blah from message_reference where ordering between a and b

This is going to be a lot quicker than selecting all the refs in the queue then just using the first x, especially when there are million messages in the queue, since for some dbs/drivers they will get the entire resultset before sending back.

I tested this last night with mysql. I successfully sent 1000000 2K messages to a queue - this works fine.

However when receiving the loads are quite slow when there are a lot of messages for the above reason.

I guess we can limit this by using db specific stuff which allows you to only select the first x messages - but I'd like to avoid db specific stuff.

Any suggestions?

[adrian.brock]

Speak to the hibernate guys.

[adrian.brock]

The usual trick is to remember the "last key" and unions.
But I don't know how portable unions, both semantically
and in terms of performance?

e.g. if you know the last message in the queue was 27
you do

select blah from message_reference where priority = 1 and msgid > 27
union
select blah from message_reference where priority = 2 and msgid > 27
etc.

This is also only going to scale well if the result set
doesn't try to load the entire database into memory
(like mysql does),

[adrian.brock]

I wrote the union thing that way (missed the order by :-)
because the alternative to unions is multiple queries.

[apwalker]

In general terms even if your not using a full blown Hibernate implementation and you want to avoid db specific stuff SQL you can use the Hibernate Dialect classes to generate the appropriate SQL.

[timfox]

There is an interesting implication to this message cancellation stuff that I do not fully understand.

If it is possible for messages to change their position in the queue due to be cancelled due to two or more JTA transactions rolling back in a different order, then how can we satisfy the JMS ordering guarantee whereby messages from a session should be received in the same order they were sent.

Doesn't make sense to me.

[adrian.brock]

You don't need to support ordering when a message is redelivered.

Doing so would be very unperformant:
http://jira.jboss.com/jira/browse/JBAS-2444

[timfox]

Not having to support ordering after failure would make my life easier

I looked through the spec trying to find where it says the message ordering guarantee doesn't apply in redelivery but couldn't find it.

The nearest I can find is:

4.4.11
A session?s recover method is used to stop a session and restart it with its first
unacknowledged message. In effect, the session?s series of delivered messages
is reset to the point after its last acknowledged message. The messages it now
delivers may be different from those that were originally delivered due to
message expiration and the arrival of higher-priority messages.

But this only mentions expiration or higher priority messages which is not what is happening in our case.

Is this mentioned somewhere else?

[adrian.brock]

Think about it. How you can guarantee the delivery order
when there are multiple sessions/transactions nacking messages.
It is entirely dependent on the timing of the clients' operations.

You could even have

client 1: nack message1
client 2: nack message2
client 2: receive gets message1
client 1: receive gets message2

where the messages delivered aren't even preserved let alone
the ordering. :-)

This does not apply to your quote, which is about client acknowledge.
The messages are "nacked" into the session (waiting for a recover() or
a subsequent ack), not the queue for client acknowledge.
So nobody else can see them unless the session is closed.

[timfox]

Agreed, it would be hard to guarantee ordering without imposing some kind of serialization of access to the channel from sessions/transactions.

So I guess then we can imply it is ok then to relax the guarantee in this situation since it might have an "unreasonable" effect on performance in order to maintain it.