Transaction does not get committed or rollback on the slave

[qren]

I am using JBoss4.0.5.GA and JBossTS4.2.2. I have a ServerA and ServerB and a client. here is my flow senario:

1. client call a stateless session BeanA on ServerA,
2. BeanA do some pre work,
3. BeanA then call another stateless session BeanB on serverB,
4. BeanB do some work,
5. BeanA do some post work.

the result shows that if either step 2 or step 5 failed, then the whole transaction get rollbacked correctly. but if step 4 failed, then transaction on ServerA rollback, but the transaction on ServerB does not get rollbacked or committed, we can see the debug information says the xa datasource transaction get started but never get rollbacked. and the database connection remains in use, and when we run the test several times, all the database connections will used up.

I hope some experts can help about this.

[marklittle]

Did you install the JTS or the JTA component of JBossTS?

[qren]

I setup JBoss 4.0.5GA using WebStart installer, choose ejb3 and "all" configurations and downloaded JBossTS 4.2.2, installed it according to the documents.

[marklittle]

As the installation instructions say http://labs.jboss.com/file-access/default/members/jbosstm/downloads/4.2.2/notes/jts_install.txt, use the tar or zip version of JBossAS.

[qren]

Thanks for reply. but how do I use ejb3 then?

[peterj]

Download the source tar.gz file and build it using a 5.0 JDK. You will get the standard zip distribution and a second version with EJB3 support. Install JTA or JTS to the EJB3 version. Worked for me on 4.0.5.

[kconner]

Peter is correct in that you can use ejb3 if you recompile the AS using JDK5. I'm surprised you were able to run any ejb3 code without doing this though as the aspect library contains dependencies on JBossTM.

While this configuration is not officially supported, and there is at least one known issue with JBossCache/Hibernate interactions, we will do our best to provide support via the forums. If you can provide a test case for your issue then we can certainly help.

[qren]

A test case is detailed in this post: http://www.jboss.org/index.html?module=bb&op=viewtopic&t=97533. If you like, I can send you a email with the code attached.

[kconner]

Apologies, I had missed that posting.

You state in that posting that the distributed transaction works, in that all data is rolled back, but that the connections are not released. In this one you state that the second server does not receive the rollback/commit.

Are these the same test cases?

Thanks,
Kev

[qren]

Yes, they are the same test case, and the same problem. when an exception happens at the slave server, the slave server does not get rollback. the transaction just hang there. so the database connection does not get released. eventually the database connection get used up, the system becomes unavailable. I can see the master server has following messages:

Executing XA statement: XA START
Executing XA statement: XA END
Executing XA statement: XA PREPARE
Executing XA statement: XA COMMIT.

But on the slave server, I only see XA START, no other three outputs when exception happens at the slave server.

If the exception happens at the master server either at the beginning of the transaction or at the end of transaction, the JTS works perfectly.

[kconner]

Okay thanks.

I'll try and setup some machines to replicate this test and have a look at what is happening

Thanks,
Kev.

[peterj]

qren,

I ran the test application you sent me. Here is my setup:

Server1:
* Windows Vista (RTM)
* Sun JDK 1.5.0_10
* JBoss AS 4.0.5.GA
* JBoss JTS 4.2.2 (not JTA!)
* MySQL 5.0.11

Server 2:
* Fedora Core 6 (kernel 2.6.18, build 2896)
* Sun JDK 1.5.0_07
* JBoss AS 4.0.5.GA
* JBoss JTS 4.2.2 (not JTA!)
* MySQL 5.0.18-pro

JTS was installed according to the ReadMe that accompanies it, with the changes as noted at http://www.jboss.com/index.html?module=bb&op=viewtopic&t=97104.

I ran both the the one and two server tests, and both passed. The three server test is going to take me a little longer to set up, I will post the results once I have them.

[qren]

Hi, Peter,

Thank you for looking into this. Can you try to increase the NUMBER_OF_TIMES_TO_RUN to 500 and then test again? The code I sent to you only have a value of 17. maybe that's still within you available connections.

You can run it during the night, so that you don't need to sit there to see if it pass or not.

Or you can just run the testRunSeveralTimesForRecord3OnBFailedToInsert for 500 times, to see if you can pass it?

[peterj]

qren,

I will give that a try next week and let you know the results.

[peterj]

Here is what I did and my results.

I set up a script to run with NUMBER_OF_TIMES_TO_RUN set to 20, 40, ... 200. The 20 runs ok, the 40 run got an error on testRunSeveralTimesForRecord3OnBFailedToInsert, after which all tests failed. I swapped my servers, so that Vista hosted serverB and FC6 hosted serverA, just to make sure that there was not a Linux of MySQL version issue. In this case, 20 and 40 ran fine, but 60 got the same error.

The client reports a RollbackException, which I think is not helpful. The server (serverB) reported:

2007-01-17 10:30:16,854 DEBUG [org.hibernate.jdbc.ConnectionManager] opening JDBC connection
2007-01-17 10:30:16,861 WARN [org.jboss.resource.connectionmanager.JBossManagedConnectionPool] Throwable while attempting to get a new connection: null
org.jboss.resource.JBossResourceException: Could not create connection; - nested throwable: (com.mysql.jdbc.exceptions.MySQLNonTransientConnectionException: Too many connections)
 at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.createManagedConnection(XAManagedConnectionFactory.java:164)
 at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.createConnectionEventListener(InternalManagedConnectionPool.java:565)
 at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:250)
 at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:529)
 at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:341)
 at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:301)
...

I did increase the max connections in the *-ds.xml file to 300, and from jmx-console it appears that only 100 connections were in use. I then increased the max connections for mySQL to 300 and ran again. This time I got up to 100 before I got the error (and jmx-console showed 300 database connections in use).

It does appear that under some circumstances (an abort?) that connections on the slave system (serverB in this case) are not being freed, and thus you can easily run out of connections. Since the user-written code is simply EJBs using declarative transactions, it would appear that either the application server or the transaction manager is not properly closing connections under certain circumstances.