1 2 Previous Next 18 Replies Latest reply on Sep 26, 2012 3:22 PM by Bjørn Konestabo

    Messages and transactions

    Bjørn Konestabo Newbie

      We're using Hornetq embedded. During import of data we create objects we put into a database. Some of the data needs to be processed later, so we fire off a message to a queue with the relevant data.

       

      My problem is that this message arrives before the database transaction has been commited, so when the consumer looks for the data to process, it's not there yet.

       

      I create the jms session transacted, but it doesn't seem to do what I expect it to do.

       

      How can I synchronize the messages and the database transaction such that the messages are delivered after the commit?

        • 1. Re: Messages and transactions
          Andy Taylor Master

          firstly I'm assuming that the messaging sending and the database update are part of the same tx using XA, if not can you post some code so i can see what you are doing, however if the assumption is correct:

           

          This a common misconception as to how Transaction Managers work, all the TM ensures is that both resources within the tx are consistent, either both commited or both rolled back. The TM knows nothing about where the resources are or what they are, it basically just calls commit on each resource in turn, usually on the order they are enlisted (altho this is not part of the spec). You can try enlisting them in a different order to make sure the database gets commited, or try a user tx, or add a deliver delay, or change the design of your application.

          1 of 1 people found this helpful
          • 2. Re: Messages and transactions
            Bjørn Konestabo Newbie

            Thanks for the clarification. We were not using XA.

             

            From what you're writing I suspect the simplest way to get what I want here is to use bean managed transactions and transacted messages, and roll back all messages if the database transaction fails. However, this still leaves the case where the database transaction is successful and the message transaction fails for some reason, which is unfortunate, but in the context of our application, it's not likely, nor critical.

             

            We are moving from a home growns messaging system to HornetQ, and the home grown used the same database, so there was no need to keep transactions in sync.

            • 3. Re: Messages and transactions
              Andy Taylor Master

              From what you're writing I suspect the simplest way to get what I want here is to use bean managed transactions and transacted messages, and roll back all messages if the database transaction fails. However, this still leaves the case where the database transaction is successful and the message transaction fails for some reason, which is unfortunate, but in the context of our application, it's not likely, nor critical.

              You can use XA to make sure that this doesnt happen, also note that you will still have the issue where the message send is commited before the database commit. I think it is possible using the jboss transaction manager to order commits but you would need to check the jboss tm docs for this.We are

              moving from a home growns messaging system to HornetQ, and the home grown used the same database, so there was no need to keep transactions in sync.

              Great to have another convert, feel free to use the forums for any issues you may come across

              • 4. Re: Messages and transactions
                Bjørn Konestabo Newbie

                I've turned on XA. Throwing a RuntimeException in the MDB onMessage-method correctly causes the message send to fail, and a re-delivery attempt is triggered soon after.

                However, the database transaction is NOT being rolled back, and I'm struggling to figure out why. Any ideas?

                • 5. Re: Messages and transactions
                  Andy Taylor Master

                  you shouldnt throw an exception from within the onMessage, its viwed as bad practice, you should handle all errors in your MDB like so

                   

                  @Resource

                     MessageDrivenContext ctx;

                   

                     public void onMessage(final Message message)

                     {

                        try

                        {

                           //do something

                        }

                        catch (JMSException e)

                        {

                           ctx.setRollbackOnly();

                        }

                     }

                   

                  If the DB connection still isnt being rolled back then its either not an XA datasource or its misconfigured

                  1 of 1 people found this helpful
                  • 6. Re: Messages and transactions
                    Bjørn Konestabo Newbie

                    Great! Calling ctx.setRollbackOnly caused an exception an revealed that I wasn't using ContainerManaged transactions as I thought I was.

                    • 7. Re: Messages and transactions
                      Bjørn Konestabo Newbie

                      It looks like it works, but unfortunately always rolling back database transaction and the message transaction at the same time is too limiting in some cases.

                       

                      Can I get both transactions bean-managed?

                      • 8. Re: Messages and transactions
                        Andy Taylor Master

                        It looks like it works, but unfortunately always rolling back database transaction and the message transaction at the same time is too limiting in some cases.

                        That doesn't make any sense, the whole point of a transaction is to handle multiple resources, in this instance there is only one transaction with 2 branches, the database and message consumption.

                        • 9. Re: Messages and transactions
                          Bjørn Konestabo Newbie

                          Only when using XA?

                           

                          I have a case where I want to consume the message despite that the database transaction fails to commit.

                          Depending on the result of the database commit I want to send other messages.

                          • 10. Re: Messages and transactions
                            Andy Taylor Master

                            Only when using XA?

                             

                            I have a case where I want to consume the message despite that the database transaction fails to commit.

                            Depending on the result of the database commit I want to send other messages.

                            yes, if you only want the consume the message as part of a tx then you dont need a tx at all. If you want DB stuff and sending messages then you have those as part of your transaction (xa)

                            • 11. Re: Messages and transactions
                              Bjørn Konestabo Newbie

                              I'm able to get what I want just by using BMT and just doing everything in separate transactions.

                              This way I can be sure if everything is persisted before I put the message on the queue.

                               

                              There is a case where queuing the message could fail and I'd ideally like to roll back the database, but it's a rare case, and fixable in our system.

                               

                              I don't want to start dealing with nested transactions right now.

                              • 12. Re: Messages and transactions
                                Andy Taylor Master

                                what scenario do you have where the consumption can roll back but the DB update and the send can still happen, seems a bit non intuitive to me

                                • 13. Re: Messages and transactions
                                  Bjørn Konestabo Newbie

                                  This is the scenario:

                                   

                                  I receive a message containing some data to persist.

                                   

                                  I store the data in the database

                                   

                                  I send a message containing some details about the persisted data to be processed by another consumer.

                                   

                                  alternatively: the database persist fails and I send data to a failed-queue.

                                   

                                  Wrapping all of this in a XA with CMT doesn't do what I want.

                                  If i use BMT I can isolate the steps, but there is still a case where sending the message after a successfull database transaction could fail.

                                  There is no case where I want to roll back the consumption.

                                  • 14. Re: Messages and transactions
                                    Andy Taylor Master

                                    Wrapping all of this in a XA with CMT doesn't do what I want.

                                    If i use BMT I can isolate the steps, but there is still a case where sending the message after a successfull database transaction could fail.

                                    If the Transaction Manager is using JTA for its BMT, which i think it does it will be using XA under the covers, so the DB update and the send will be atomic.

                                     

                                    There is no case where I want to roll back the consumption.

                                    So failing to update the DB and sending the message can fail but the consumption of the message shouldnt, if thats what you want just set the redelivery count to 0 and use CMT

                                    1 2 Previous Next