Flow Transaction Question

[fouquet.f]

Hi ,

I made  a crash test of the Fuse ESB server.

When I activate XA transaction between my JMS internals points I don't loose any message. But I have a very poor performance 7 minutes for 3000 msg cons 50 s without XA.

When I remove theses transactions Fuse seems to loose somes messages.

So i see here that the default Flow method use a persistant NMR but no transaction between component exchange.

How configure transaction for internal JBI container exchange of message and not loosing msg in case of serveur crash ?

Regards

[mielket]

Hi again,

Can you please briefly describe your application scenario and how the jms components are used?

Also, please paste your relevant configuration.

[fouquet.f]

Hi again ,

So my test configuration is like this shema

JMSTOPIC --> JMSCOnsumer1XA --> POJO(made JDBC) --> JMSPRovider2 --> JMSTOPIC2 --> JMSCOnsumer2XA --> POJO(made JDBC) --> JMSPRovider3 --> JMSTOPIC3 --> JMSConsumer3XA --> POJO --> FileSender

So I would made persistance point with JMSTopic Msg are sending Persistant and the subscrition are durable too.

I would like to made Transaction in the Pojo. In other terme I msg could not be suppressed from a JMS point if there is not an ack from the insertion in the next JMS point

For the moment I had the default config for Fuse Broker and ESB and for the JMS consumer I had made the xbean.xml with the information of our previous thread.

Thanks a lot for your help

Regards

[mielket]

Do you perhaps have a simple testcase that reproduces the problem? I am still slightly unsure how your application works.

As part of Forum ID761 I ran some performance tests using the new jms endpoint and also turned on XA transactions but did not see any such bad performance that you describe here.

A little testcase that reproduces your problem would be very helpful in analyzing your issue.

[mielket]

Also I am attaching my testcase that I used for Forum ID 761. You can simply build and deploy it into SMX and then use any JMS client to send messages to queue "queue.id760-New". It will be picked up by the JMS consumer and send to a bean inside a XA transaction.

You can turn on/off XA transaction in servicemix-jms-perf-test-jms-consumer-su\src\main\resources\xbean.xml by setting either

transacted="xa" or transacted="none".

[fouquet.f]

Hi ,

Ok I will try your SA immediatly

For testing performances I have a POJO who build 3000 messages from a database and send it to the topic ( very small messages )

To simulate it you juste have to made a loop ( I will send you the archive )

With XA i have 50 s delai , without I have average 10 .

Do you think JCA ou throoling will be better ?

Regards

[fouquet.f]

Hi again,

So I send you a service assembly

who send 3000 messages on the Queue design by your SA

So you just have to deploy the to SA and call the http://localhost:8192/test/ to call the pojo

on my side i have 10 secondes without XA and more than two minutes with XA

I monitor the CPU % with JConsole

Any idea ?

Edited by: fouquet.f on Apr 7, 2009 4:44 PM

Edited by: fouquet.f on Apr 7, 2009 4:44 PM

[fouquet.f]

Perhaps another idea

I have a lot of disk writing when using Xa or JMS , is it possible to configure transaction manager to use someting else ?

Regards

[fouquet.f]

Another information

I have trace ActiveMQ with the previous example and activate log4j to DEBUG for ActiveMQ

and I had the following result , I'm very surprising to found rollback instruction ???

Perhaps there is a problem on the timeout ?

Just an example

09:39:40,632 | DEBUG | DefaultMessageListenerContainer-1 | ActiveMQSession          | pache.activemq.ActiveMQSession  501 | ID:Q407047-1509-1239175821248-2:23:1 Transaction Commit :TX:ID:Q407047-1509-1239175821248-2:23:1615

09:39:40,632 | DEBUG | InactivityMonitor ReadCheck | InactivityMonitor        | .transport.InactivityMonitor$1   72 | 17459 ms elapsed since last read check.

09:39:40,648 | DEBUG | InactivityMonitor ReadCheck | InactivityMonitor        | .transport.InactivityMonitor$1   81 | Aborting read check.. Not enough time elapsed since last read check.

09:39:40,648 | DEBUG | ActiveMQ Transport: tcp:///127.0.0.1:1547 | AMQMessageStore          | vemq.store.amq.AMQMessageStore  216 | Journalled transacted message remove for: ID:Q407047-1509-1239175821248-2:109:1:1:189, at: offset = 1854627, file = 1, size = 341, type = 1

09:39:40,695 | DEBUG | ActiveMQ Transport: tcp:///127.0.0.1:1547 | AMQMessageStore          | mq.store.amq.AMQMessageStore$3  229 | Transacted message remove commit for: ID:Q407047-1509-1239175821248-2:109:1:1:189, at: offset = 1854627, file = 1, size = 341, type = 1

09:39:40,695 | DEBUG | ActiveMQ Task | AMQMessageStore          | vemq.store.amq.AMQMessageStore  375 | Doing batch update... adding: 0 removing: 1

09:39:40,695 | DEBUG | DefaultMessageListenerContainer-1 | ActiveMQSession          | pache.activemq.ActiveMQSession  501 | ID:Q407047-1509-1239175821248-2:277:1 Transaction Commit :null

09:39:40,695 | DEBUG | ActiveMQ Task | AMQMessageStore          | vemq.store.amq.AMQMessageStore  413 | Batch update done.

09:39:40,695 | DEBUG | DefaultMessageListenerContainer-1 | ActiveMQSession          | pache.activemq.ActiveMQSession  521 | ID:Q407047-1509-1239175821248-2:277:1 Transaction Rollback

09:39:40,695 | DEBUG | DefaultMessageListenerContainer-1 | ActiveMQMessageConsumer  | tivemq.ActiveMQMessageConsumer  432 | ID:Q407047-1509-1239175821248-2:23:1:1 received message: MessageDispatch {commandId = 0, responseRequired = false, consumerId = ID:Q407047-1509-1239175821248-2:23:1:1, destination = queue://queue.id760-New, message = ActiveMQTextMessage {commandId = 193, responseRequired = true, messageId = ID:Q407047-1509-1239175821248-2:112:1:1:189, originalDestination = null, originalTransactionId = null, producerId = ID:Q407047-1509-1239175821248-2:112:1:1, destination = queue://queue.id760-New, transactionId = null, expiration = 0, timestamp = 1239176053071, arrival = 0, brokerInTime = 1239176053103, brokerOutTime = 1239176249703, correlationId = null, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = null, marshalledProperties = org.apache.activemq.util.ByteSequence@65d8d2, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {org.apache.servicemix.body=}, redeliveryCounter = 0}

09:39:40,726 | DEBUG | InactivityMonitor WriteCheck | InactivityMonitor        | .transport.InactivityMonitor$2   99 | 9995 ms elapsed since last write check.

09:39:40,726 | INFO  | pool-flow.seda.servicemix-bean-thread-30 | MyBean                   | com.progress.cs.id760.MyBean     37 | Received exchange: InOnly[

  id: ID:10.67.153.138-12084a27c5e-7:1615

  status: Active

  role: provider

  service: Pojo

  endpoint: endpoint

  in: }, redeliveryCounter = 0}

09:39:40,820 | INFO  | pool-flow.seda.servicemix-bean-thread-32 | MyBean                   | com.progress.cs.id760.MyBean     37 | Received exchange: InOnly[

  id: ID:10.67.153.138-12084a27c5e-7:1616

  status: Active

  role: provider

  service: Pojo

  endpoint: endpoint

  in: }, redeliveryCounter = 0}

09:39:40,913 | INFO  | pool-flow.seda.servicemix-bean-thread-35 | MyBean                   | com.progress.cs.id760.MyBean     37 | Received exchange: InOnly[

  id: ID:10.67.153.138-12084a27c5e-7:1617

  status: Active

  role: provider

  service: Pojo

  endpoint: endpoint

  in:

]

[mielket]

Regarding your testcase first: I cannot deploy it into FUSE ESB. It raises:

Unrecognized xbean namespace mapping: http://jencks.org/amqpool/2.0

Your xbean.xml of ZTestPerfJMSProvide explicitly sets up the <amqpool:xa-pool> but the corresponding pom.xml does not define a dependency to

<dependency>
   <groupId>org.jencks</groupId>
   <artifactId>jencks-amqpool</artifactId>
   <version>2.1</version>
</dependency>

Unfortunately I cannot recompile your testcase as it is unable to resolve dev.capgemini.com:dam.frontal.servicemix:pom:0.0.1-SNAPSHOT

from any known repository.

How did you get this SA to deploy? Did you manually change the FUSE ESB classpath? I tried adding jencks-amqpool-2.0.jar manually to my FUSE ESB classpath but that resides in other problems during startup.

[fouquet.f]

Hi ,

Sorry for my POM.xml who was made to much quicly

So I have add the jencks-2.1.jar and jencks-amqpool-2.0.jar to my {fuse-install}/lib path and it work like that .

But you can made the same SA on your side, it is very simple i have juste a bean triggered by something ( HTTP or quartz ) who send 3000 msg on the queue.

How can I help you to found the problem ?

As i said on my previous message i have see rollback instruction in JMS LOG and the XA transaction is very usefull for my project. So your help is very usefull for me .

Thanks by advance

Regards

[mielket]

I got your testcase to compile and added the dependency to jencks-amqpool in the JMS component. With that change it now deploys fine.

I can also run it fine here but don't get to see that much of a difference between XA and non-XA.

Please notice, an XA transaction will always add a significant overhead over plain message consumption.

[fouquet.f]

Ok ,

and do you use same version of jencks than me ?

Do you use last fuse ESB 3.4 ?

Have you some transaction rollback in your log ?

I'm very surprising if you don't have same result can your bus read the 3000 msg in less than 1 minutes ?

Thanks by advance

[fouquet.f]

I have made another test today with our test case and I have more than 3minutes ...

I have join the log file

I have count 9000 commit and 3000 rollback in the log

How many time on your side ?

Edited by: fouquet.f on Apr 8, 2009 4:25 PM

[mielket]

Lets take a step back on this issue.

Firstly I got to realize this morning the testcase you sent me (ZTestPerf-SA.zip) does not use any XA transaction. None of the components in the testcase start a XA transaction and when I was running it, I did not see any commit or rollback in servicemix.log happening.

So I reverted back to my testcase that I attached here two days ago (servicemix-jms-perf-test). As it starts with a JMS consumer, I can set transacted="xa" on the JMS consumer endpoint configuration. So this testcase does use transactions.

Using my testcase I ran some performance tests and noticed a roughly 50% performance degradation when transactions are turned on (with 1000 msgs processed in each test run). That seems rather reasonable to me and is even less than what I expected.

I am running all my tests on FUSE ESB 3.4.0.1-fuse and use the Jencks transaction manager that is setup by FUSE ESB during startup.

Also, my log file does currently not show any transaction rollbacks.

The higher disk usage when transactions are turned on are explained by each tx commit resulting a write to persistence layer.

Edited by: tmielke on Apr 9, 2009 1:46 PM