6 Replies Latest reply on Nov 4, 2010 7:36 AM by Mark Little

    Could not enlist resource in distributed tx

    Arbi Sookazian Master

      We have the following code in production that we can't replicate the behavior for in non-prod envmts.  We are using Oracle 10g in all envmts with JBoss 4.2.2.GA.  barDao and bazDao are using two different datasources: one is local-tx-datasource and the other is xa-datasource (I know that both should be XA in a distributed tx but that's what it is now in prod).  We are getting the "could not enlist resource in tx" error when the 2nd line is executed (this is the local-tx-datasource).  We are unable to reproduce using the same EAR in non-prod envmt.  Why?  There is no ejb-jar.xml transactional semantics override happening.  In a different method of another SLSB which is a distrubted tx as well, the behavior is not happening.  This is extremely confusing.

       

      @Stateless
      public class FooProcessorServiceBean {

       

                public void fooMethod() {

       

                     barDao.fetch();

                     bazDao.fetch();

       

                }

       

      }

       

      All db operations via JPA/Hibernate are read-only (this is a reporting app).  We can use @NOT_SUPPORTED tx attribute type for the above method but we can't reproduce to see if this fixes the problem in another envmt other than prod.

       

      What config files should I check (I've looked at oracle-ds.xml and jboss-service.xml)?  Is it possible this is data-related?  thx.

       

      2010-11-01 12:41:47,568 WARN [org.hibernate.util.JDBCExceptionReporter] SQL  Error: 0, SQLState: null 
      2010-11-01 12:41:47,568 ERROR  [org.hibernate.util.JDBCExceptionReporter] Could not enlist in transaction on  entering meta-aware object!; - nested throwable:  (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist  resource, see the previous warnings. tx=TransactionImple < ac, BasicAction:  a02451c:9925:4cc0c841:7c7fc status: ActionStatus.ABORT_ONLY >); - nested  throwable: (org.jboss.resource.JBossResourceException: Could not enlist in  transaction on entering meta-aware object!; - nested throwable:  (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist  resource, see the previous warnings. tx=TransactionImple < ac, BasicAction:  a02451c:9925:4cc0c841:7c7fc status: ActionStatus.ABORT_ONLY >)) 
      2010-11-01 12:41:47,569 INFO  [org.hibernate.event.def.DefaultLoadEventListener] Error performing load command 
      org.hibernate.exception.GenericJDBCException: Cannot open connection 
              at  org.hibernate.exception.SQLStateConverter.handledNonSpecificException(SQLStateConverter.java:103) 
              at  org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:91) 
              at  org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
        • 1. Re: Could not enlist resource in distributed tx
          Arbi Sookazian Master

          Is XA datasource required if the db's are on the same Oracle server?  What if they're on different Oracle servers?

          • 2. Re: Could not enlist resource in distributed tx
            Andrew Dinn Master

            Arbi Sookazian wrote:

             

            We have the following code in production that we can't replicate the behavior for in non-prod envmts.


              . . .

             

            2010-11-01 12:41:47,568 WARN Could not enlist resource in distributed tx SQL  Error: 0, SQLState: null

            2010-11-01 12:41:47,568 ERROR  Could not enlist resource in distributed tx Could not enlist in transaction on  entering meta-aware object!; - nested throwable:  (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist  resource, see the previous warnings. tx=TransactionImple &lt; ac, BasicAction:  a02451c:9925:4cc0c841:7c7fc status: ActionStatus.ABORT_ONLY &gt;); - nested  throwable: (org.jboss.resource.JBossResourceException: Could not enlist in  transaction on entering meta-aware object!; - nested throwable:  (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist  resource, see the previous warnings.

             

              . . .

             

            Your transaction's status has been set to ABORT_ONLY. This can happen for a variety of reasons. There is a hint in the error message which you might want to follow in order to identify or discount some of those reasons -- I've outlined it in bold, just in case. So, check your log to see if you can find any prior messages explaining why setRollbackOnly was called on the transaction. If you don't find any messages then you may need to resort to switching on tracing for the TX code. Alternatively, you might be able to use Byteman just to trace calls to setRollbackOnly.

            • 3. Re: Could not enlist resource in distributed tx
              Arbi Sookazian Master

              I turned on lower logging levels for arjuna and hibernate packages and here is the result.  Note that I used @TransactionAttribute(value=TransactionAttributeType.NOT_SUPPORTED) and it had no change in behavior (I reproduced the error).

               

              2010-11-03 15:12:24,954 DEBUG [org.hibernate.impl.SessionImpl] opened session at timestamp: 5279016324931584
              2010-11-03 15:12:24,954 TRACE [org.hibernate.jdbc.JDBCContext] TransactionFactory reported no active transaction; Synchronization not registered
              2010-11-03 15:12:24,954 DEBUG [org.hibernate.ejb.AbstractEntityManagerImpl] Looking for a JTA transaction to join
              2010-11-03 15:12:24,954 DEBUG [org.hibernate.jdbc.JDBCContext] successfully registered Synchronization
              2010-11-03 15:12:24,954 TRACE [org.hibernate.ejb.AbstractEntityManagerImpl] Adding flush() and close() synchronization
              2010-11-03 15:12:24,955 DEBUG [org.hibernate.ejb.AbstractEntityManagerImpl] Looking for a JTA transaction to join
              2010-11-03 15:12:24,955 DEBUG [org.hibernate.ejb.AbstractEntityManagerImpl] Transaction already joined
              2010-11-03 15:12:24,958 TRACE [org.hibernate.event.def.DefaultLoadEventListener] loading entity: [com.echo.cis.domain.model.Merchant#37616]
              2010-11-03 15:12:24,958 TRACE [org.hibernate.event.def.DefaultLoadEventListener] attempting to resolve: [com.echo.cis.domain.model.Merchant#37616]
              2010-11-03 15:12:24,959 TRACE [org.hibernate.event.def.DefaultLoadEventListener] object not resolved in any cache: [com.echo.cis.domain.model.Merchant#37616]
              2010-11-03 15:12:24,959 TRACE [org.hibernate.persister.entity.AbstractEntityPersister] Fetching entity: [com.echo.cis.domain.model.Merchant#37616]
              2010-11-03 15:12:24,959 DEBUG [org.hibernate.loader.Loader] loading entity: [com.echo.cis.domain.model.Merchant#37616]
              2010-11-03 15:12:24,959 DEBUG [org.hibernate.jdbc.AbstractBatcher] about to open PreparedStatement (open PreparedStatements: 0, globally: 0)
              2010-11-03 15:12:24,959 DEBUG [org.hibernate.jdbc.ConnectionManager] opening JDBC connection
              2010-11-03 15:12:24,963 WARN  [com.arjuna.ats.jta.logging.loggerI18N] [com.arjuna.ats.internal.jta.transaction.arjunacore.lastResource.disallow] [com.arjuna.ats.internal.jta.transaction.arjunacore.lastReso
              urce.disallow] Adding multiple last resources is disallowed. Current resource is org.jboss.resource.connectionmanager.TxConnectionManager$LocalXAResource@4d183cfb
              2010-11-03 15:12:24,971 DEBUG [org.hibernate.util.JDBCExceptionReporter] Cannot open connection [???]
              org.jboss.util.NestedSQLException: Could not enlist in transaction on entering meta-aware object!; - nested throwable: (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist resource, s
              ee the previous warnings. tx=TransactionImple < ac, BasicAction: -53ec2673:aefc:4cd1dde7:31 status: ActionStatus.ABORT_ONLY >); - nested throwable: (org.jboss.resource.JBossResourceException: Could not enl
              ist in transaction on entering meta-aware object!; - nested throwable: (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist resource, see the previous warnings. tx=TransactionImple <
              ac, BasicAction: -53ec2673:aefc:4cd1dde7:31 status: ActionStatus.ABORT_ONLY >))

              • 4. Re: Could not enlist resource in distributed tx
                Arbi Sookazian Master

                btw, there are two identical PUs deployed in the same EAR, I wonder what negative effect, if any, that may have?

                 

                from jmx-console:

                 

                • ear=com.echo.reports.ear-1.5.2-SNAPSHOT.ear,jar=com.echo.billing.reports.dao.gemini-1.5.2-SNAPSHOT.jar,unitName=reports-persistence-unit
                • ear=com.echo.reports.ear-1.5.2-SNAPSHOT.ear,jar=com.echo.billing.reports.services.persistence-1.5.2-SNAPSHOT.jar,unitName=reports-persistence-unit
                • 5. Re: Could not enlist resource in distributed tx
                  Michael Musgrove Master

                  Arbi Sookazian wrote:


                  [com.arjuna.ats.internal.jta.transaction.arjunacore.lastResource.disallow] [com.arjuna.ats.internal.jta.transaction.arjunacore.lastReso

                  urce.disallow] Adding multiple last resources is disallowed. Current resource is org.jboss.resource.connectionmanager.TxConnectionManager$LocalXAResource@4d183cfb
                  2010-11-03 15:12:24,971 DEBUG [org.hibernate.util.JDBCExceptionReporter] Cannot open connection [???]
                  org.jboss.util.NestedSQLException: Could not enlist in transaction on entering meta-aware object!; - nested throwable: (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist resource, s
                  ee the previous warnings. tx=TransactionImple < ac, BasicAction: -53ec2673:aefc:4cd1dde7:31 status: ActionStatus.ABORT_ONLY >); - nested throwable: (org.jboss.resource.JBossResourceException: Could not enl
                  ist in transaction on entering meta-aware object!; - nested throwable: (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist resource, see the previous warnings. tx=TransactionImple <
                  ac, BasicAction: -53ec2673:aefc:4cd1dde7:31 status: ActionStatus.ABORT_ONLY >))

                  This snippet is saying that a second local resouce is being enlisted in the transaction. Only one non 2PC aware resource is allowed to join the transaction. There is an article on the wiki that covers this scenario: http://community.jboss.org/wiki/Multiple1PC

                  • 6. Re: Could not enlist resource in distributed tx
                    Mark Little Master

                    You can override the default to enable multiple 1PC-aware resources (I can't recall the option at the moment, but it'll be in the docs). However, this is a really really really bad thing to do. A bit like crossing the streams from proton packs (http://en.wikipedia.org/wiki/Proton_pack): not to be done unless in dire circumstances and then all bets are off.