Messages and transactions

[bjornko]

We're using Hornetq embedded. During import of data we create objects we put into a database. Some of the data needs to be processed later, so we fire off a message to a queue with the relevant data.

My problem is that this message arrives before the database transaction has been commited, so when the consumer looks for the data to process, it's not there yet.

I create the jms session transacted, but it doesn't seem to do what I expect it to do.

How can I synchronize the messages and the database transaction such that the messages are delivered after the commit?

[ataylor]

firstly I'm assuming that the messaging sending and the database update are part of the same tx using XA, if not can you post some code so i can see what you are doing, however if the assumption is correct:

This a common misconception as to how Transaction Managers work, all the TM ensures is that both resources within the tx are consistent, either both commited or both rolled back. The TM knows nothing about where the resources are or what they are, it basically just calls commit on each resource in turn, usually on the order they are enlisted (altho this is not part of the spec). You can try enlisting them in a different order to make sure the database gets commited, or try a user tx, or add a deliver delay, or change the design of your application.

[bjornko]

Thanks for the clarification. We were not using XA.

From what you're writing I suspect the simplest way to get what I want here is to use bean managed transactions and transacted messages, and roll back all messages if the database transaction fails. However, this still leaves the case where the database transaction is successful and the message transaction fails for some reason, which is unfortunate, but in the context of our application, it's not likely, nor critical.

We are moving from a home growns messaging system to HornetQ, and the home grown used the same database, so there was no need to keep transactions in sync.

[ataylor]

From what you're writing I suspect the simplest way to get what I want here is to use bean managed transactions and transacted messages, and roll back all messages if the database transaction fails. However, this still leaves the case where the database transaction is successful and the message transaction fails for some reason, which is unfortunate, but in the context of our application, it's not likely, nor critical.

You can use XA to make sure that this doesnt happen, also note that you will still have the issue where the message send is commited before the database commit. I think it is possible using the jboss transaction manager to order commits but you would need to check the jboss tm docs for this.We are

moving from a home growns messaging system to HornetQ, and the home grown used the same database, so there was no need to keep transactions in sync.

Great to have another convert, feel free to use the forums for any issues you may come across

[bjornko]

I've turned on XA. Throwing a RuntimeException in the MDB onMessage-method correctly causes the message send to fail, and a re-delivery attempt is triggered soon after.

However, the database transaction is NOT being rolled back, and I'm struggling to figure out why. Any ideas?

[ataylor]

you shouldnt throw an exception from within the onMessage, its viwed as bad practice, you should handle all errors in your MDB like so

@Resource

   MessageDrivenContext ctx;

   public void onMessage(final Message message)

   {

      try

      {

         //do something

      }

      catch (JMSException e)

      {

         ctx.setRollbackOnly();

      }

   }

If the DB connection still isnt being rolled back then its either not an XA datasource or its misconfigured

[bjornko]

Great! Calling ctx.setRollbackOnly caused an exception an revealed that I wasn't using ContainerManaged transactions as I thought I was.

[bjornko]

It looks like it works, but unfortunately always rolling back database transaction and the message transaction at the same time is too limiting in some cases.

Can I get both transactions bean-managed?

[ataylor]

It looks like it works, but unfortunately always rolling back database transaction and the message transaction at the same time is too limiting in some cases.

That doesn't make any sense, the whole point of a transaction is to handle multiple resources, in this instance there is only one transaction with 2 branches, the database and message consumption.

[bjornko]

Only when using XA?

I have a case where I want to consume the message despite that the database transaction fails to commit.

Depending on the result of the database commit I want to send other messages.

[ataylor]

Only when using XA?

I have a case where I want to consume the message despite that the database transaction fails to commit.

Depending on the result of the database commit I want to send other messages.

yes, if you only want the consume the message as part of a tx then you dont need a tx at all. If you want DB stuff and sending messages then you have those as part of your transaction (xa)

[bjornko]

I'm able to get what I want just by using BMT and just doing everything in separate transactions.

This way I can be sure if everything is persisted before I put the message on the queue.

There is a case where queuing the message could fail and I'd ideally like to roll back the database, but it's a rare case, and fixable in our system.

I don't want to start dealing with nested transactions right now.

[ataylor]

what scenario do you have where the consumption can roll back but the DB update and the send can still happen, seems a bit non intuitive to me

[bjornko]

This is the scenario:

I receive a message containing some data to persist.

I store the data in the database

I send a message containing some details about the persisted data to be processed by another consumer.

alternatively: the database persist fails and I send data to a failed-queue.

Wrapping all of this in a XA with CMT doesn't do what I want.

If i use BMT I can isolate the steps, but there is still a case where sending the message after a successfull database transaction could fail.

There is no case where I want to roll back the consumption.

[ataylor]

Wrapping all of this in a XA with CMT doesn't do what I want.

If i use BMT I can isolate the steps, but there is still a case where sending the message after a successfull database transaction could fail.

If the Transaction Manager is using JTA for its BMT, which i think it does it will be using XA under the covers, so the DB update and the send will be atomic.

There is no case where I want to roll back the consumption.

So failing to update the DB and sending the message can fail but the consumption of the message shouldnt, if thats what you want just set the redelivery count to 0 and use CMT