9 Replies Latest reply on Jun 8, 2012 9:37 AM by markaddleman

    Intermittent transaction errors

    gamvi01

      Hi Steve,

       

      What does below exceptions standfor ? we are seeing the below errors randomly.

      1) org.teiid.jdbc.TeiidSQLException: Transaction already associated with request. at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:113) at org.teiid.jdbc.TeiidSQLException.create(TeiidSQLException.java:70) at org.teiid.jdbc.StatementImpl.postReceiveResults(StatementImpl.java:618) at org.teiid.jdbc.StatementImpl.access$100(StatementImpl.java:62) at org.teiid.jdbc.StatementImpl$2.onCompletion(StatementImpl.java:533) at

       

       

      2) Transaction is not active:

      tx=TransactionImple < ac, BasicAction: -7235bee1:c0d9:4fcca430:c3 status: ActionStatus.ABORT_ONLY >; - nested throwable: (javax.resource.ResourceException: Transaction is not active: tx=TransactionImple < ac, BasicAction: -7235bee1:c0d9:4fcca430:c3 status: ActionStatus.ABORT_ONLY >). null. Error Code=0

       

       

       

       

        • 1. Re: Intermittent transaction errors
          shawkins

          Vineela,

           

          1) May occur if you are using the TeiidDriver or a non-XA managed TeiidDataSource with a local connection and a transaction already exists on the thread.  There will add an option in 8.1 for teiid to use any thread bound transaction it finds for execution with a local connection, but I'm not sure that will make it into 7.7.1.  If you are not using socket connections, or you are using a managed TeiidDataSource, then there is an issue and we'd need more information to track it down.

           

          2) That may be due to an earlier issue, which you would hopefully see earlier in the log, or if you are using transaction timeouts, the transaction may have already been invalidated due to timeout.

           

          Steve

          • 2. Re: Intermittent transaction errors
            markaddleman

            Based off the following log entries:

            java.lang.AssertionError: Transaction already associated with request.

                at org.teiid.core.util.Assertion.failed(Assertion.java:73)

                at org.teiid.core.util.Assertion.assertTrue(Assertion.java:68)

                at org.teiid.dqp.internal.process.Request.createProcessor(Request.java:342)

                at org.teiid.dqp.internal.process.Request.processRequest(Request.java:464)

                at org.teiid.dqp.internal.process.RequestWorkItem.processNew(RequestWorkItem.java:520)

                at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:277)

                at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:49)

                at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:216)

                at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:240)

                at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:122)

                at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:292)

                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)

                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)

                at java.lang.Thread.run(Thread.java:769)

            java.sql.SQLException: You cannot commit during a managed transaction!

                at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnection.jdbcCommit(BaseWrapperManagedConnection.java:659)

                at org.jboss.resource.adapter.jdbc.WrappedConnection.commit(WrappedConnection.java:486)

            07 Jun 2012 20:59:40,080 GMT WARN  [com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Worker138_QueryProcessorQueue30129) [com.arjuna.ats.arjuna.coordinator.BasicAction_57] - Commit of action id -7235bef5:2602:4fd10910:df invoked while multiple threads active within it.

            07 Jun 2012 20:59:40,081 GMT WARN  [com.arjuna.ats.arjuna.logging.arjLoggerI18N] (Worker138_QueryProcessorQueue30129) [com.arjuna.ats.arjuna.coordinator.CheckedAction_1] - CheckedAction::check - atomic action -7235bef5:2602:4fd10910:df commiting with 2 threads active!

            07 Jun 2012 20:59:58,654 GMT ERROR [org.teiid.PROCESSOR] (Worker138_QueryProcessorQueue30159) Unexpected exception for request dqjBj0loaKo3.446

            java.lang.AssertionError: Transaction already associated with request.

            I suspect that Teiid connection objects can't be shared across threads.  More precisely, connection objects can't be enlisted in two different transactions at once.  Normally, an application wouldn't do this but since we need to maintain Teiid temporary tables across user requests, we are stuffing the connection object into the web session.  Obviously, this opens us up to the potential that the same connection will be involved in multiple transactions.

             

            If you think this is the problem, I've got two questions:

            1. Do you think using XA connections will avoid the problem?  We're planning to switch to XA connections to make use of Teiid local connections
            2. I think you mentioned there's a JIRA ticket regarding global temp tables but I couldn't find it.  Is there one?
            • 3. Re: Intermittent transaction errors
              markaddleman

              Looks like our posts crossed each other.  I'm pretty sure we need to switch to XA transactions.  We're planning that in the coming weeks so know we'll if there is a deeper problem then.

               

              Thanks

              • 4. Re: Intermittent transaction errors
                rareddy

                Using XAConnection for Teiid, will enable the Teiid participate in the on-going transaction that is started by your application, but it is still *not*  safe to share connection object.

                • 5. Re: Intermittent transaction errors
                  shawkins

                  Mark,

                   

                  Txn issues were mentioned in https://issues.jboss.org/browse/TEIID-1865  We may need to understand this more about what you are trying to do to know what is going on. Are you creating competing local connections (for example calling Connection.setAutoCommit(false)) in concurrent threads?  This would be susceptible to a race condition if un-synchronized, but if you had a consistent notion of a unit-of-work across threads, then adding synchronization around the start would prevent starting a txn when one is already present. 

                   

                  Or is autoCommitTxn left at the default of DETECT and we are automatically creating a transaction under the covers for concurrent requests?  If so, we're making an assumption in the engine that transactional work on a connection will be performed serially.  You could either serialize transaction requests, or if autoCommitTxns are not needed, then you could work-around by using the OFF setting. 

                   

                  Switching to XA in a typical container managed scenario could address this issue if you set the pool to track by transaction, then usage of that connection under a transaction will be serialized.  So the question here would be is the perfermance hit too great?

                   

                  As for global temporary tables, I couldn't find the issue either.  Can you create one?  The major issues with implementing that feature with-in Teiid are with cluster safety/performance and consistency under concurrently modifying transactions.  More could be done to simplify the scenario, such as delegating to an external db.

                   

                  Steve

                  • 6. Re: Intermittent transaction errors
                    rareddy
                    As for global temporary tables, I couldn't find the issue either.  Can you create one?  The major issues with implementing that feature with-in Teiid are with cluster safety/performance and consistency under concurrently modifying transactions.  More could be done to simplify the scenario, such as delegating to an external db.

                     

                    Delegating to external would solve this issue https://issues.jboss.org/browse/TEIID-196

                    • 7. Re: Intermittent transaction errors
                      markaddleman

                      Created TEIID-2067 for global temp tables.  From our point of view, materializing the temp tables in an external database (I imagine the mechanism would work similarly to external materialization) is desirable.

                       

                      A bit of background:  The only reason we're sharing the connection object between threads is to achieve a very limited version of global temp tables.  In our web app, one user click causes a result set to materialize to a temp table.  In another set of clicks, the user is browsing through the result set.  Because temp tables are scoped to the Teiid session, we cannot use a typical connection pooling pattern.  Instead, we put the connection object into the user's web session and try to apply proper synchronization when using it. 

                       

                      From the info in TEIID-1865 (which I had forgotten), I can see that our recent use of setAutoCommit(false) is probably the culprit here.  From the data source definition, auto commit is true.  When we populate the temp tables, we turn auto commit to false.  Although I would like to turn auto commit off for the entire app, that is not feasible at the moment.  Instead, I think we can leave auto commit on when populating the temp tables. 

                       

                      The lesson here is, it's not nice to fool Mother Nature, she gets mad:  JDBC really wants connections to be thread bound. 

                       

                      If there is any information I can provide regarding TEIID-2067, please let me know.

                      • 8. Re: Intermittent transaction errors
                        rareddy

                        Looks like you are trying to store the application's data in the Teiid in form of temp table. Why not store the in application context or save locally some other way to avoid issues until the global tables feature is available.

                        • 9. Re: Intermittent transaction errors
                          markaddleman

                          Yes, that's exactly right.  Originally, we wanted to use those temp tables for more than just browsing result sets including joining to other Teiid tables so the temp table approach within Teiid seemed ideal.  Due to time constraints, we no longer need to join so keeping the data outside of Teiid becomes possible (although, undesirable).