2 Replies Latest reply on Sep 18, 2007 7:12 AM by Mark Little

    Unfixable XA error after JTA failure

    Tony Green Newbie

      I am working on XA configuration for an application which consists of OLTP plus auditing. The whole stack consists of :

      jboss 4.0.2
      db2
      spring / with tx aop
      spring webflow
      hibernate

      Application code is deployed essentially in two components one for the web side and one for the data side. At the present stage of development I have all of my audit tables deployed into the OLTP schema with two separate -ds.xml files. In the future these will point to separate schemas.

      I have my spring jta configuration working to the extent that I can show correct propagation behaviour.

      I tried testing rollback behaviour by deliberately throwing an UnsupportedOperationException from the middle of the transaction and I now have my system in an unusable state: whenever I try to run the same transaction code again I get a RollbackException ('already marked for rollback...'). The problem persists over restarts of the database and app server.

      I cannot find an explanation anywhere of how the distributed transaction is persisted and how I could clean up its state.

      What I actually see in more detail is this:

      1) 17:25:49,287 WARN [TransactionImpl] XAException: tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=G00214/16, BranchQual=, localId=16] errorCode=XAER_NOTA
      com.ibm.db2.jcc.c.te: XAER_NOTA
      at com.ibm.db2.jcc.a.bc.a(bc.java:1828)
      at com.ibm.db2.jcc.a.bc.a(bc.java:2144)
      ... and then back up through jboss TransactionImpl and various connection manager classes, and then the Spring hibernate3.LocalDatasourceConnectionProvider and back through hibernate, the application Daos to the web application stuff.

      2) 17:25:49,287 INFO [TxConnectionManager] Could not enlist in transaction on entering meta-aware object!
      javax.transaction.SystemException: enlistResource failed
      at org.jboss.resource.connectionmanager.TxConnectionManager$
      TxConnectionEventListener.enlist(TxConnectionManager.java:446)

      3) 17:25:49,302 WARN [TransactionImpl] XAException: tx=TransactionImpl:XidImpl[FormatId=257, GlobalId=G00214/16, BranchQual=, localId=16] erro
      rCode=XAER_PROTO
      com.ibm.db2.jcc.c.te: XAER_PROTO
      at com.ibm.db2.jcc.a.bc.a(bc.java:1828)
      at com.ibm.db2.jcc.a.bc.a(bc.java:1719)
      at com.ibm.db2.jcc.a.bc.end(bc.java:701)
      at org.jboss.resource.adapter.jdbc.xa.XAManagedConnection.end(XAManagedConnection.java:146)

      4) 17:25:49,302 WARN [TxConnectionManager] Unchecked throwable in managedConnectionDisconnected()
      javax.resource.ResourceException: Failure to delist resource
      at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener.delist(TxConnectionManager.java:477)
      at org.jboss.resource.connectionmanager.TxConnectionManager.managedConnectionDisconnected(TxConnectionManager.java:337)

      5) 17:25:49,318 INFO [DefaultLoadEventListener] Error performing load command
      org.hibernate.exception.GenericJDBCException: Cannot open connection
      at org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:103)
      at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:91)
      at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)

      6) 17:25:49,365 INFO [TxConnectionManager] Could not enlist in transaction on entering meta-aware object!
      javax.transaction.SystemException: Could not get XAResource from ManagedConnection!javax.transaction.RollbackException: Already marked for r
      ollback TransactionImpl:XidImpl[FormatId=257, GlobalId=G00214/16, BranchQual=, localId=16]
      at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener.enlist(TxConnectionManager.java:441)
      at org.jboss.resource.connectionmanager.TxConnectionManager.managedConnectionReconnected(TxConnectionManager.java:322)

      7) 17:25:49,365 INFO [TxConnectionManager] Could not enlist in transaction on entering meta-aware object!
      javax.transaction.SystemException: Could not get XAResource from ManagedConnection!javax.transaction.RollbackException: Already marked for
      ollback TransactionImpl:XidImpl[FormatId=257, GlobalId=G00214/16, BranchQual=, localId=16]
      at org.jboss.resource.connectionmanager.TxConnectionManager$TxConnectionEventListener.enlist(TxConnectionManager.java:441)
      at org.jboss.resource.connectionmanager.TxConnectionManager.managedConnectionReconnected(TxConnectionManager.java:322)

      So a whole bunch of compoents there failing to make sense of the situation.

      You may like to know I have the datasources configured like this (angle brackets replaced with round ones to defeat escaper):

      (xa-datasource)
      (jndi-name)jdbc/as(/jndi-name)
      (xa-datasource-class)com.ibm.db2.jcc.DB2XADataSource(/xa-datasource-class)
      (xa-datasource-property name="DriverType")4(/xa-datasource-property)
      (xa-datasource-property name="DatabaseName")[...omitted...](/xa-datasource-property)
      (xa-datasource-property name="User")[...omitted...](/xa-datasource-property)
      (xa-datasource-property name="Password")[...omitted...](/xa-datasource-property)
      (xa-datasource-property name="PortNumber")50000(/xa-datasource-property)
      (xa-datasource-property name="ServerName")localhost(/xa-datasource-property)
      (xa-datasource-property name="CurrentSchema")[...omitted...](/xa-datasource-property)
      (transaction-isolation)TRANSACTION_READ_UNCOMMITTED(/transaction-isolation)
      (min-pool-size)40(/min-pool-size)
      (max-pool-size)40(/max-pool-size)
      (track-statements)true(/track-statements)
      (track-connection-by-tx)true(/track-connection-by-tx)
      (no-tx-separate-pools/)
      (prepared-statement-cache-size)10(/prepared-statement-cache-size)
      (set-tx-query-timeout/)
      (/xa-datasource)

      And then this datasource is imported into Spring with:

      (bean id="dataSourceAudit" class="org.springframework.jndi.JndiObjectFactoryBean")
      (property name="jndiName")
      (value)java:comp/env/jdbc/as(/value)
      (/property)
      (/bean)

      (bean id="sessionFactoryAudit" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean")
      (property name="dataSource" ref="dataSourceAudit" /)
      (property name="mappingResources")
      (list merge="true")
      (value)[...omitted....].hbm.xml(/value)
      (/list)
      (/property)
      (property name="hibernateProperties")
      (props)
      (prop key="hibernate.transaction.factory_class")org.hibernate.transaction.JTATransactionFactory(/prop)
      (prop key="hibernate.connection.release_mode")on_close(/prop)
      (prop key="hibernate.dialect")org.hibernate.dialect.DB2Dialect(/prop)
      (prop key="hibernate.show_sql")false(/prop)
      (prop key="hibernate.default_schema")[...omitted...](/prop)
      (prop key="hibernate.cache.provider_class")org.hibernate.cache.EhCacheProvider(/prop)
      (/props)
      (/property)
      (/bean)

      And the final link is :

      (bean id="txManager" class="org.springframework.transaction.jta.JtaTransactionManager" )
      (/bean)
      (tx:annotation-driven transaction-manager="txManager"/)

      So the questions are:

      1) How do clean up my existing system?

      2) What did I do wrong?

      3) Should I be configuring any of the XA recovery stuff I've seen mentioned?

      Huge thanks in advance,
      Tony