1 2 Previous Next 16 Replies Latest reply on Apr 9, 2009 10:39 AM by Torsten Mielke

    Flow Transaction Question

    François FOUQUET Newbie

      Hi ,

       

      I made  a crash test of the Fuse ESB server.

       

      When I activate XA transaction between my JMS internals points I don't loose any message. But I have a very poor performance 7 minutes for 3000 msg cons 50 s without XA.

       

      When I remove theses transactions Fuse seems to loose somes messages.

       

      So i see here that the default Flow method use a persistant NMR but no transaction between component exchange.

       

      How configure transaction for internal JBI container exchange of message and not loosing msg in case of serveur crash ?

       

      Regards

        • 1. Re: Flow Transaction Question
          Torsten Mielke Apprentice

          Hi again,

           

          Can you please briefly describe your application scenario and how the jms components are used?

           

          Also, please paste your relevant configuration.

          • 2. Re: Flow Transaction Question
            François FOUQUET Newbie

            Hi again ,

             

            So my test configuration is like this shema

             

             

            JMSTOPIC --> JMSCOnsumer1XA --> POJO(made JDBC) --> JMSPRovider2 --> JMSTOPIC2 --> JMSCOnsumer2XA --> POJO(made JDBC) --> JMSPRovider3 --> JMSTOPIC3 --> JMSConsumer3XA --> POJO --> FileSender

             

            So I would made persistance point with JMSTopic Msg are sending Persistant and the subscrition are durable too.

             

            I would like to made Transaction in the Pojo. In other terme I msg could not be suppressed from a JMS point if there is not an ack from the insertion in the next JMS point

             

             

            For the moment I had the default config for Fuse Broker and ESB and for the JMS consumer I had made the xbean.xml with the information of our previous thread.

             

             

            Thanks a lot for your help

             

            Regards

            • 3. Re: Flow Transaction Question
              Torsten Mielke Apprentice

              Do you perhaps have a simple testcase that reproduces the problem? I am still slightly unsure how your application works.

              As part of Forum ID761 I ran some performance tests using the new jms endpoint and also turned on XA transactions but did not see any such bad performance that you describe here.

               

              A little testcase that reproduces your problem would be very helpful in analyzing your issue.

              • 4. Re: Flow Transaction Question
                Torsten Mielke Apprentice

                Also I am attaching my testcase that I used for Forum ID 761. You can simply build and deploy it into SMX and then use any JMS client to send messages to queue "queue.id760-New". It will be picked up by the JMS consumer and send to a bean inside a XA transaction.

                 

                You can turn on/off XA transaction in servicemix-jms-perf-test-jms-consumer-su\src\main\resources\xbean.xml by setting either

                transacted="xa" or transacted="none".

                • 5. Re: Flow Transaction Question
                  François FOUQUET Newbie

                  Hi ,

                   

                  Ok I will try your SA immediatly

                   

                  For testing performances I have a POJO who build 3000 messages from a database and send it to the topic ( very small messages )

                   

                  To simulate it you juste have to made a loop ( I will send you the archive )

                   

                  With XA i have 50 s delai , without I have average 10 .

                   

                  Do you think JCA ou throoling will be better ?

                   

                  Regards

                  • 6. Re: Flow Transaction Question
                    François FOUQUET Newbie

                    Hi again,

                     

                    So I send you a service assembly

                    who send 3000 messages on the Queue design by your SA

                     

                    So you just have to deploy the to SA and call the http://localhost:8192/test/ to call the pojo

                     

                    on my side i have 10 secondes without XA and more than two minutes with XA

                     

                    I monitor the CPU % with JConsole

                     

                    Any idea ?

                     

                    Edited by: fouquet.f on Apr 7, 2009 4:44 PM

                     

                    Edited by: fouquet.f on Apr 7, 2009 4:44 PM

                    • 7. Re: Flow Transaction Question
                      François FOUQUET Newbie

                      Perhaps another idea

                       

                      I have a lot of disk writing when using Xa or JMS , is it possible to configure transaction manager to use someting else ?

                       

                      Regards

                      • 8. Re: Flow Transaction Question
                        François FOUQUET Newbie

                        Another information

                         

                        I have trace ActiveMQ with the previous example and activate log4j to DEBUG for ActiveMQ

                         

                        and I had the following result , I'm very surprising to found rollback instruction ???

                        Perhaps there is a problem on the timeout ?

                         

                         

                         

                        Just an example

                         

                         

                        09:39:40,632 | DEBUG | DefaultMessageListenerContainer-1 | ActiveMQSession          | pache.activemq.ActiveMQSession  501 | ID:Q407047-1509-1239175821248-2:23:1 Transaction Commit :TX:ID:Q407047-1509-1239175821248-2:23:1615

                        09:39:40,632 | DEBUG | InactivityMonitor ReadCheck | InactivityMonitor        | .transport.InactivityMonitor$1   72 | 17459 ms elapsed since last read check.

                        09:39:40,648 | DEBUG | InactivityMonitor ReadCheck | InactivityMonitor        | .transport.InactivityMonitor$1   81 | Aborting read check.. Not enough time elapsed since last read check.

                        09:39:40,648 | DEBUG | ActiveMQ Transport: tcp:///127.0.0.1:1547 | AMQMessageStore          | vemq.store.amq.AMQMessageStore  216 | Journalled transacted message remove for: ID:Q407047-1509-1239175821248-2:109:1:1:189, at: offset = 1854627, file = 1, size = 341, type = 1

                        09:39:40,695 | DEBUG | ActiveMQ Transport: tcp:///127.0.0.1:1547 | AMQMessageStore          | mq.store.amq.AMQMessageStore$3  229 | Transacted message remove commit for: ID:Q407047-1509-1239175821248-2:109:1:1:189, at: offset = 1854627, file = 1, size = 341, type = 1

                        09:39:40,695 | DEBUG | ActiveMQ Task | AMQMessageStore          | vemq.store.amq.AMQMessageStore  375 | Doing batch update... adding: 0 removing: 1

                        09:39:40,695 | DEBUG | DefaultMessageListenerContainer-1 | ActiveMQSession          | pache.activemq.ActiveMQSession  501 | ID:Q407047-1509-1239175821248-2:277:1 Transaction Commit :null

                        09:39:40,695 | DEBUG | ActiveMQ Task | AMQMessageStore          | vemq.store.amq.AMQMessageStore  413 | Batch update done.

                        09:39:40,695 | DEBUG | DefaultMessageListenerContainer-1 | ActiveMQSession          | pache.activemq.ActiveMQSession  521 | ID:Q407047-1509-1239175821248-2:277:1 Transaction Rollback

                        09:39:40,695 | DEBUG | DefaultMessageListenerContainer-1 | ActiveMQMessageConsumer  | tivemq.ActiveMQMessageConsumer  432 | ID:Q407047-1509-1239175821248-2:23:1:1 received message: MessageDispatch {commandId = 0, responseRequired = false, consumerId = ID:Q407047-1509-1239175821248-2:23:1:1, destination = queue://queue.id760-New, message = ActiveMQTextMessage {commandId = 193, responseRequired = true, messageId = ID:Q407047-1509-1239175821248-2:112:1:1:189, originalDestination = null, originalTransactionId = null, producerId = ID:Q407047-1509-1239175821248-2:112:1:1, destination = queue://queue.id760-New, transactionId = null, expiration = 0, timestamp = 1239176053071, arrival = 0, brokerInTime = 1239176053103, brokerOutTime = 1239176249703, correlationId = null, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = null, marshalledProperties = org.apache.activemq.util.ByteSequence@65d8d2, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {org.apache.servicemix.body=}, redeliveryCounter = 0}

                        09:39:40,726 | DEBUG | InactivityMonitor WriteCheck | InactivityMonitor        | .transport.InactivityMonitor$2   99 | 9995 ms elapsed since last write check.

                        09:39:40,726 | INFO  | pool-flow.seda.servicemix-bean-thread-30 | MyBean                   | com.progress.cs.id760.MyBean     37 | Received exchange: InOnly[

                          id: ID:10.67.153.138-12084a27c5e-7:1615

                          status: Active

                          role: provider

                          service: Pojo

                          endpoint: endpoint

                          in: }, redeliveryCounter = 0}

                        09:39:40,820 | INFO  | pool-flow.seda.servicemix-bean-thread-32 | MyBean                   | com.progress.cs.id760.MyBean     37 | Received exchange: InOnly[

                          id: ID:10.67.153.138-12084a27c5e-7:1616

                          status: Active

                          role: provider

                          service: Pojo

                          endpoint: endpoint

                          in: }, redeliveryCounter = 0}

                        09:39:40,913 | INFO  | pool-flow.seda.servicemix-bean-thread-35 | MyBean                   | com.progress.cs.id760.MyBean     37 | Received exchange: InOnly[

                          id: ID:10.67.153.138-12084a27c5e-7:1617

                          status: Active

                          role: provider

                          service: Pojo

                          endpoint: endpoint

                          in:

                        ]

                        • 9. Re: Flow Transaction Question
                          Torsten Mielke Apprentice

                          Regarding your testcase first: I cannot deploy it into FUSE ESB. It raises:

                          Unrecognized xbean namespace mapping: http://jencks.org/amqpool/2.0

                           

                          Your xbean.xml of ZTestPerfJMSProvide explicitly sets up the <amqpool:xa-pool> but the corresponding pom.xml does not define a dependency to

                          <dependency>
                             <groupId>org.jencks</groupId>
                             <artifactId>jencks-amqpool</artifactId>
                             <version>2.1</version>
                          </dependency>
                          

                           

                          Unfortunately I cannot recompile your testcase as it is unable to resolve dev.capgemini.com:dam.frontal.servicemix:pom:0.0.1-SNAPSHOT

                           

                          from any known repository.

                           

                          How did you get this SA to deploy? Did you manually change the FUSE ESB classpath? I tried adding jencks-amqpool-2.0.jar manually to my FUSE ESB classpath but that resides in other problems during startup.

                          • 10. Re: Flow Transaction Question
                            François FOUQUET Newbie

                            Hi ,

                             

                            Sorry for my POM.xml who was made to much quicly

                             

                            So I have add the jencks-2.1.jar and jencks-amqpool-2.0.jar to my {fuse-install}/lib path and it work like that .

                             

                            But you can made the same SA on your side, it is very simple i have juste a bean triggered by something ( HTTP or quartz ) who send 3000 msg on the queue.

                             

                            How can I help you to found the problem ?

                            As i said on my previous message i have see rollback instruction in JMS LOG and the XA transaction is very usefull for my project. So your help is very usefull for me .

                             

                            Thanks by advance

                            Regards

                            • 11. Re: Flow Transaction Question
                              Torsten Mielke Apprentice

                              I got your testcase to compile and added the dependency to jencks-amqpool in the JMS component. With that change it now deploys fine.

                              I can also run it fine here but don't get to see that much of a difference between XA and non-XA.

                              Please notice, an XA transaction will always add a significant overhead over plain message consumption.

                              • 12. Re: Flow Transaction Question
                                François FOUQUET Newbie

                                Ok ,

                                and do you use same version of jencks than me ?

                                Do you use last fuse ESB 3.4 ?

                                Have you some transaction rollback in your log ?

                                 

                                I'm very surprising if you don't have same result can your bus read the 3000 msg in less than 1 minutes ?

                                 

                                Thanks by advance

                                • 13. Re: Flow Transaction Question
                                  François FOUQUET Newbie

                                  I have made another test today with our test case and I have more than 3minutes ...

                                   

                                  I have join the log file

                                   

                                  I have count 9000 commit and 3000 rollback in the log

                                   

                                  How many time on your side ?

                                   

                                  Edited by: fouquet.f on Apr 8, 2009 4:25 PM

                                  • 14. Re: Flow Transaction Question
                                    Torsten Mielke Apprentice

                                    Lets take a step back on this issue.

                                     

                                    Firstly I got to realize this morning the testcase you sent me (ZTestPerf-SA.zip) does not use any XA transaction. None of the components in the testcase start a XA transaction and when I was running it, I did not see any commit or rollback in servicemix.log happening.

                                     

                                    So I reverted back to my testcase that I attached here two days ago (servicemix-jms-perf-test). As it starts with a JMS consumer, I can set transacted="xa" on the JMS consumer endpoint configuration. So this testcase does use transactions.

                                     

                                    Using my testcase I ran some performance tests and noticed a roughly 50% performance degradation when transactions are turned on (with 1000 msgs processed in each test run). That seems rather reasonable to me and is even less than what I expected.

                                     

                                    I am running all my tests on FUSE ESB 3.4.0.1-fuse and use the Jencks transaction manager that is setup by FUSE ESB during startup.

                                     

                                    Also, my log file does currently not show any transaction rollbacks.

                                     

                                    The higher disk usage when transactions are turned on are explained by each tx commit resulting a write to persistence layer.

                                     

                                    Edited by: tmielke on Apr 9, 2009 1:46 PM

                                    1 2 Previous Next