JBossDeveloper

I had written a test yesterday that would cause an out-of-order issue, and I have just committed it (PagingTest::disabled_testDepageDuringTransaction2)

The test will send a message before paging. sending 4 messages when the system enters into page. and another 5 with the system out of paging, and validating the order at the end. (it is failing with out of order now).

Clebert Suconic wrote:

 

I had written a test yesterday that would cause an out-of-order issue, and I have just committed it (PagingTest::disabled_testDepageDuringTransaction2)

 

The test will send a message before paging. sending 4 messages when the system enters into page. and another 5 with the system out of paging, and validating the order at the end. (it is failing with out of order now).

fiy, I re-applied your patch locally and it fixes the failing test you added PagingTest::disabled_testDepageDuringTransaction2.

I slightly modified the patch (i.e. making explicit that the paging uses a "copy" of the transaction without its TransactionOperations).

With this patch, messages which are routed after system went out of paging will be correctly routed with a transaction context.

I'm running the full test suite to check for regression and will commit the fix today (or on monday).

As mentioned before, I would like to see a proper explanation of the problem before any patches are applied.

the issue is related to paging and transaction.

it occurs when the paging mode of a destination changes while messages are sent in a transaction

- we send a message in a tx to an adress

- we send multiple non-transacted messages to the address

  => the address starts to page

- we send other messages in the tx

  => messages are added to a PageOperation executed when the tx will commit

- we consume non-transacted messages from the queue

  => the address starts to depage

- we commit the tx

  => when the PageOperations are executed, the address is no longer paging and messages

       will be routed by the post office *without* the transactional context in PageOperation.beforeCommit

  => these messages will be routed before the 1st message sent in the tx is routed (done in AddOperation.afterCommit)

  => out of order issue

with clebert's patch, we make sure that messages routed from PageOperations when the address is not paging will keep the transaction context so they remain in order.

Wouldn't a simpler fix be, if any parts of a transaction are paged, then all the transaction must be paged, irrespective of whether the address leaves paging?

When we Page on TX, we defer routing / paging at commit time. During commit time the message could leave page state, and we can't page at that point.

Even if we add method to page in blocks, or route in blocks (page everything or nothing, or route everything or nothing as you said), we could still have a case where the system won't be paging during commit and we would need to add messages back to the current transaction through routing.

i.e. Some way to change the current TX still needed.

Clebert Suconic wrote:

... and we would need to add messages back to the current transaction through routing.

Why? Why not always page the messages even if the address is not paging any more. I.e. if at least one message in the tx is paged they all must be paged.

There's a caveat on paging... when the address is in page mode, everything must be paged up to the moment it is not paging any more.

Say, if we page when there are no more messages on the address. you could have a situation where nothing will be depaged. We don't have any threads watching for the sizes.. it all happens based on the size of the Address during ACK.

During the ACK we do a check:

if sizeOfQueue - maxSizeQueue < pageSize => depage

Having the possibility of paging when the address is empty would break depage.

Can't you just prompt a depage as soon as the tx commits?

Tim Fox wrote:

 

Can't you just prompt a depage as soon as the tx commits?

I would need to think about it. I have a bad feeling on races when having multiple producers and consumers.

The page system was made to be depaged during ACK.

It would be easy to have race conditions when having multiple producers and consumers doing it transactionally.  I would need to spend some time on testing it. It wouldn't be an easier change at the end.