MysqlXAException: XAER_OUTSIDE

[sandello]

I'm developing a EJB3 project with ejb3-persistence and several connections to DB (mysql 5.0.37, Connector/J 5.0.7).
Yesterday I migrated from jboss 4.0.5 to 4.2.1 I changed my datasource description from local-tx-datasource to xa-datasource. My JUnit test failed with exception:

15:27:00,586 WARN [loggerI18N] [com.arjuna.ats.internal.jta.transaction.arjunacore.enliststarterror] [com.arjuna.ats.internal.jta.transaction.arjunacore.enliststarterror] TransactionImple.enlistResource - XAResource.start returned: XAException.XAER_OUTSIDE for < 131075, 27, 25, 1-a000247:a806:46fb6f94:16da000247:a806:46fb6f94:174 >
15:27:00,586 ERROR [STDERR] com.mysql.jdbc.jdbc2.optional.MysqlXAException: XAER_OUTSIDE: Some work is done outside global transaction
15:27:00,587 ERROR [STDERR] at com.mysql.jdbc.jdbc2.optional.MysqlXAConnection.mapXAExceptionFromSQLException(MysqlXAConnection.java:564)
15:27:00,587 ERROR [STDERR] at com.mysql.jdbc.jdbc2.optional.MysqlXAConnection.dispatchCommand(MysqlXAConnection.java:547)
15:27:00,587 ERROR [STDERR] at com.mysql.jdbc.jdbc2.optional.MysqlXAConnection.start(MysqlXAConnection.java:488)
15:27:00,587 ERROR [STDERR] at com.mysql.jdbc.jdbc2.optional.SuspendableXAConnection.start(SuspendableXAConnection.java:126)
15:27:00,587 ERROR [STDERR] at org.jboss.resource.adapter.jdbc.xa.XAManagedConnection.start(XAManagedConnection.java:121)
15:27:00,587 ERROR [STDERR] at org.jboss.resource.connectionmanager.xa.JcaXAResourceWrapper.start(JcaXAResourceWrapper.java:113)
15:27:00,587 ERROR [STDERR] at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:774)
15:27:00,587 ERROR [STDERR] at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.enlistResource(TransactionImple.java:497)
15:27:00,587 ERROR [STDERR] at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener$TransactionSynchronization.enlist(TxConnectionManager.java:773)
15:27:00,587 ERROR [STDERR] at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener.enlist(TxConnectionManager.java:564)
15:27:00,587 ERROR [STDERR] at org.jboss.resource.connectionmanager.TxConnectionManager.managedConnectionReconnected(TxConnectionManager.java:337)
15:27:00,587 ERROR [STDERR] at org.jboss.resource.connectionmanager.BaseConnectionManager2.reconnectManagedConnection(BaseConnectionManager2.java:518)
15:27:00,588 ERROR [STDERR] at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:399)
15:27:00,588 ERROR [STDERR] at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:842)
15:27:00,588 ERROR [STDERR] at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:88)
15:27:00,588 ERROR [STDERR] at org.hibernate.ejb.connection.InjectedDataSourceConnectionProvider.getConnection(InjectedDataSourceConnectionProvider.java:47)
15:27:00,588 ERROR [STDERR] at org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:423)
15:27:00,588 ERROR [STDERR] at org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:144)
15:27:00,588 ERROR [STDERR] at org.hibernate.jdbc.AbstractBatcher.prepareQueryStatement(AbstractBatcher.java:139)
15:27:00,588 ERROR [STDERR] at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1547)
15:27:00,588 ERROR [STDERR] at org.hibernate.loader.Loader.doQuery(Loader.java:673)
15:27:00,588 ERROR [STDERR] at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:236)
15:27:00,588 ERROR [STDERR] at org.hibernate.loader.Loader.doList(Loader.java:2220)
15:27:00,588 ERROR [STDERR] at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2104)
15:27:00,588 ERROR [STDERR] at org.hibernate.loader.Loader.list(Loader.java:2099)
15:27:00,588 ERROR [STDERR] at org.hibernate.loader.criteria.CriteriaLoader.list(CriteriaLoader.java:94)
15:27:00,588 ERROR [STDERR] at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1569)
15:27:00,588 ERROR [STDERR] at org.hibernate.impl.CriteriaImpl.list(CriteriaImpl.java:283)
15:27:00,588 ERROR [STDERR] at com.mycompany.proj.util.storage.dataselector.CriteriaSelector.selectEntities(CriteriaSelector.java:95)
15:27:00,588 ERROR [STDERR] at com.mycompany.proj.util.storage.dataselector.CriteriaSelector.selectEntities(CriteriaSelector.java:69)
15:27:00,588 ERROR [STDERR] at com.mycompany.proj.util.storage.dataselector.CriteriaSelector.selectSingleEntities(CriteriaSelector.java:99)
15:27:00,588 ERROR [STDERR] at com.mycompany.proj.util.storage.dataselector.SimpleSelector.selectSingleEntities(SimpleSelector.java:64)
15:27:00,588 ERROR [STDERR] at com.mycompany.proj.util.storage.dataselector.UriSelector.selectSingleEntities(UriSelector.java:80)
15:27:00,589 ERROR [STDERR] at com.mycompany.proj.util.storage.BSSelectorNew.selectSingleEntity(BSSelectorNew.java:53)
15:27:00,589 ERROR [STDERR] at com.mycompany.proj.admin.storage.RoleStorage.com$telecom_m$billingtm$admin$storage$RoleStorage$save$aop(RoleStorage.java:61)
15:27:00,589 ERROR [STDERR] at com.mycompany.proj.admin.storage.RoleStorage$save_N67176538923528357.invokeNext(RoleStorage$save_N67176538923528357.java)
15:27:00,589 ERROR [STDERR] at com.mycompany.proj.kernel.cms.CMSAspect.update(CMSAspect.java:64)
15:27:00,589 ERROR [STDERR] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
15:27:00,589 ERROR [STDERR] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
15:27:00,589 ERROR [STDERR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
15:27:00,589 ERROR [STDERR] at java.lang.reflect.Method.invoke(Method.java:597)
15:27:00,589 ERROR [STDERR] at org.jboss.aop.advice.PerInstanceAdvice.invoke(PerInstanceAdvice.java:121)
15:27:00,589 ERROR [STDERR] at com.mycompany.proj.admin.storage.RoleStorage$save_N67176538923528357.invokeNext(RoleStorage$save_N67176538923528357.java)
15:27:00,589 ERROR [STDERR] at com.mycompany.proj.admin.storage.RoleStorage.save(RoleStorage.java)
15:27:00,589 ERROR [STDERR] at com.mycompany.proj.admin.server.AdminServerSEJB3.save(AdminServerSEJB3.java:178)
15:27:00,589 ERROR [STDERR] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
15:27:00,589 ERROR [STDERR] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
15:27:00,589 ERROR [STDERR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
15:27:00,589 ERROR [STDERR] at java.lang.reflect.Method.invoke(Method.java:597)
15:27:00,589 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:112)
15:27:00,589 ERROR [STDERR] at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:166)
15:27:00,589 ERROR [STDERR] at org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke(EJB3InterceptorsInterceptor.java:63)
15:27:00,589 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,589 ERROR [STDERR] at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:54)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,590 ERROR [STDERR] at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,590 ERROR [STDERR] at org.jboss.aspects.tx.TxPolicy.invokeInOurTx(TxPolicy.java:79)
15:27:00,590 ERROR [STDERR] at org.jboss.aspects.tx.TxInterceptor$RequiresNew.invoke(TxInterceptor.java:262)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,590 ERROR [STDERR] at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:76)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,590 ERROR [STDERR] at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,590 ERROR [STDERR] at org.jboss.aspects.security.AuthenticationInterceptor.invoke(AuthenticationInterceptor.java:77)
15:27:00,590 ERROR [STDERR] at org.jboss.ejb3.security.Ejb3AuthenticationInterceptor.invoke(Ejb3AuthenticationInterceptor.java:106)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,590 ERROR [STDERR] at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:46)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,590 ERROR [STDERR] at org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke(AsynchronousInterceptor.java:106)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
15:27:00,590 ERROR [STDERR] at org.jboss.ejb3.stateless.StatelessContainer.dynamicInvoke(StatelessContainer.java:278)
15:27:00,590 ERROR [STDERR] at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:106)
15:27:00,590 ERROR [STDERR] at org.jboss.aspects.remoting.AOPRemotingInvocationHandler.invoke(AOPRemotingInvocationHandler.java:82)
15:27:00,590 ERROR [STDERR] at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:734)
15:27:00,590 ERROR [STDERR] at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:560)
15:27:00,590 ERROR [STDERR] at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:369)
15:27:00,590 ERROR [STDERR] at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:165)
15:27:00,592 WARN [JDBCExceptionReporter] SQL Error: 0, SQLState: null
15:27:00,592 ERROR [JDBCExceptionReporter] Could not enlist in transaction on entering meta-aware object!; - nested throwable: (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist resource, see the previous warnings. tx=TransactionImple < ac, BasicAction: a000247:a806:46fb6f94:16d status: ActionStatus.ABORT_ONLY >); - nested throwable: (org.jboss.resource.JBossResourceException: Could not enlist in transaction on entering meta-aware object!; - nested throwable: (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist resource, see the previous warnings. tx=TransactionImple < ac, BasicAction: a000247:a806:46fb6f94:16d status: ActionStatus.ABORT_ONLY >))

I deployed modules and the only ds required for the test in question. Nothing changed, the test failed with the same exception.

Environment:
Fedora Linux 7

java version "1.6.0_01"
Java(TM) SE Runtime Environment (build 1.6.0_01-b06)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_01-b06, mixed mode)

JBoss 4.2.1
Mysql 5.0.37
Mysql Conntector/J 5.0.7

Datasource:
<xa-datasource>
<jndi-name>AdminDS</jndi-name>
<xa-datasource-property name="URL">jdbc:mysql://localhost:3306/bs_dict</xa-datasource-property>
<xa-datasource-class>com.mysql.jdbc.jdbc2.optional.MysqlXADataSource</xa-datasource-class>
<xa-datasource-property name="PinGlobalTxToPhysicalConnection">true</xa-datasource-property>
<user-name>root</user-name>


<transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
<track-connection-by-tx/>
<!--isSameRM-override-value>false</isSameRM-override-value-->

<exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter</exception-sorter-class-name>

<type-mapping>mySQL</type-mapping>


<blocking-timeout-millis>5000</blocking-timeout-millis>
</xa-datasource>

[jhalliday]

I discussed this one with MySQL's JDBC guy a while back. It's essentially due to a difference of interpretation of the JDBC specs. I'll ping MySQL and see if they decided to change their driver behaviour to match what every other database I've tested does. If not we'll have to put a workaround into the transaction manager.

[marklittle]

What's the difference in interpretation?

[jhalliday]

It relates to the way XA aware connections behave when used outside the scope of a global tx. Here is an extract of my report to MySQL:

---
In the test, I obtain a Connection via an XADataSource and use it to perform work both within and outside of an XA transaction. Where non-XA work is followed by XA work, this causes an error.

com.mysql.jdbc.jdbc2.optional.MysqlXAException: XAER_OUTSIDE: Some work is done outside global transaction
at com.mysql.jdbc.jdbc2.optional.MysqlXAConnection.mapXAExceptionFromSQLException(MysqlXAConnection.java:564)
at com.mysql.jdbc.jdbc2.optional.MysqlXAConnection.dispatchCommand(MysqlXAConnection.java:547)
at com.mysql.jdbc.jdbc2.optional.MysqlXAConnection.start(MysqlXAConnection.java:488)

The reason is that connections obtained from an XADataSource have autocommit set to false by default. The test expects true and therefore does not explicitly commit the non-XA work on the connection.

My reasoning is based on reading of the JDBC 4.0 spec. Section 10.1.1 states that the default is autocommit enabled, which indeed it is for connections obtained from non-XA datasources in the mysql driver. Section 12.4 states that when used outside of a global tx, the connections from an XADataSource should behave exactly like local tx connections. I take this to imply that they should have autocommit enabled by default. The other 5 major databases we are testing against (Oracle, MS SQL, DB2, Sybase, PostgreSQL) behave in this way so my tests pass on those.
---

However the JDBC-2.0 stdext spec can be read as requiring different behaviour. In my opinion the mysql driver is buggy, but it's not clear cut.

[marklittle]

In the Elizabethan ?Potato? episode of the BBC Blackadder TV show, Edmund Blackadder confronts the captain (Rum) of the ship he is on about the total lack of a crew for their impending voyage of discovery:

"Edmund: I was under the impression that it was common maritime practice for a ship to have a crew.

Rum: Opinion is divided on the subject.

Edmund: Oh, really? [starting to get the picture]

Rum: Yahs. All the other captains say it is; I say it isn't"

The rest, as they say, is history!

[weston.price]

There is a workaround for this in JCA, let me see if I can find the thread on MySQL. It was committed to this forum awhile ago.

[weston.price]

Try

<no-tx-separate-pools/>

in your *-ds.xml file. This will effectively force a split into two connection pools, one for connections used in a global txn and one in a non transactional manner.

[jhalliday]

MySQL are changing the behavior of their drivers to be consistent with everyone else. Until their 5.0.8 or 5.1.4 driver releases just stick with Weston's workaround.