1 2 Previous Next 16 Replies Latest reply on Nov 23, 2007 10:19 AM by Adrian Brock

    Opening connection from a marked rollback tx

    Novice

      I have a case where the transaction is marked rollback during the course of a complex process. But I still have to read from the DB and do some more processing. During that a new DB connection is opened (for read-only purposes) behind the scenes and thats where the connection manager complains:

      Caused by: javax.resource.ResourceException: Transaction is not active: tx=Trans
      actionImpl:XidImpl[FormatId=257, GlobalId=sg1317z.corproot.net/1761318, BranchQu
      al=, localId=1761318]
       at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedCo
      nnection(TxConnectionManager.java:290)
       at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateC
      onnection(BaseConnectionManager2.java:379)
       at org.jboss.resource.connectionmanager.BaseConnectionManager2$Connectio
      nManagerProxy.allocateConnection(BaseConnectionManager2.java:812)
       at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(Wrapp
      erDataSource.java:88)
       ... 188 more



      I think the connection manager should allow to use a transaction that is marked rollback because it's perfectly legal to read from the DB even if the tx is going to be rolled back.

        • 1. Re: Opening connection from a marked rollback tx
          Vicky Kak Master

           

          "oglueck" wrote:
          But I still have to read from the DB and do some more processing. During that a new DB connection is opened (for read-only purposes) behind the scenes and thats where the connection manager complains:

          Are you not able to tackle this at the application level ?


          • 2. Re: Opening connection from a marked rollback tx
            Novice

            No, that's really hard. Because the tx can be set rollback-only at many different points in the process. So I would need to basically check at every read operation and then manually suspend the tx - a lot of boilerplate code. That's too horrible. I rather patch the connection manager.

            • 3. Re: Opening connection from a marked rollback tx
              Vicky Kak Master

               

              "oglueck" wrote:
              I rather patch the connection manager.

              Have you completed the patch or this is yet to be done ?

              • 4. Re: Opening connection from a marked rollback tx
                Novice

                To be done, really. I was hoping to get a comment from a JBoss developer. Because I really think it is a bad misbehaviour.

                • 5. Re: Opening connection from a marked rollback tx
                  Vicky Kak Master

                  For your requirement you really don't need Transaction and from the logs I could make out that you are using local-tx-datasource/xa-tx-datasource .
                  If you configure no-tx-datasource things should work well , can you try this and let me know if it works ?
                  Another option which comes to my mind is to disable the connection sharing across transaction , this can be done by setting
                  <track-connection-by-tx> - whether the connection should be "locked" to the transaction, returning it to the pool at the end of the transaction (default true for Local, false for XA)
                  Read this
                  http://wiki.jboss.org/wiki/Wiki.jsp?page=ConfigJCACommon

                  • 6. Re: Opening connection from a marked rollback tx
                    Novice

                    We heavily rely on XA as multiple resource adapters are involved in a tx. Using local or no-tx connections is absolutely no option. And we already use track-connection-by-tx because Oracle connections are not capable of sharing the same connection with multiple XA transactions.

                    Actually the thing is, I don't know why a new connection is opened by Hibernate in this case and why it doesn't use an existing one. Because there should certainly be at least one free connection available that has already been used with this tx. It may be a bug in Hibernate, but I am completely unsure.

                    But what's wrong with modifying the connection manager to allow this? This should perfectly be legal as I read the specs.

                    • 7. Re: Opening connection from a marked rollback tx
                      Vicky Kak Master

                       

                      "oglueck" wrote:

                      Actually the thing is, I don't know why a new connection is opened by Hibernate in this case and why it doesn't use an existing one. Because there should certainly be at least one free connection available that has already been used with this tx. It may be a bug in Hibernate, but I am completely unsure.

                      This should be checked , if you are enabling track-connection-by-tx then every time you try to get the connection from the Pool the pool should return you the new handle to the same existing ManagedConnection.


                      But what's wrong with modifying the connection manager to allow this? This should perfectly be legal as I read the specs.

                      Playing with the CM should be the last option , it is trivial thing to do .

                      • 8. Re: Opening connection from a marked rollback tx
                        Novice

                         

                        the Pool the pool should return you the new handle to the same existing ManagedConnection.


                        Yes, that's what I thought too should happen. In fact it does most of the time. We can run for days without any problem and suddenly it occurs a couple of times in a row. The effect could depend on the load or on network conditions. Maybe the connection somehow doesn't get returned to the pool in time or something. In anycase there happens something that is not transparent to the application.

                        • 9. Re: Opening connection from a marked rollback tx
                          Vicky Kak Master

                           

                          "oglueck" wrote:
                          Maybe the connection somehow doesn't get returned to the pool in time or something.

                          Subsequent Connections are taken from the TransactionLocal and not from the Pool , here is the code
                          public ConnectionListener getConnection(Transaction trackByTransaction, Subject subject, ConnectionRequestInfo cri)
                           throws ResourceException
                           {
                           // Determine the pool key for this request
                           boolean separateNoTx = false;
                           if (noTxSeparatePools)
                           separateNoTx = clf.isTransactional();
                           Object key = getKey(subject, cri, separateNoTx);
                           SubPoolContext subPool = getSubPool(key, subject, cri);
                          
                           InternalManagedConnectionPool mcp = subPool.getSubPool();
                          
                           // Are we doing track by connection?
                           TransactionLocal trackByTx = subPool.getTrackByTx();
                          
                           // Simple case
                           if (trackByTransaction == null || trackByTx == null)
                           {
                           ConnectionListener cl = mcp.getConnection(subject, cri);
                           if (traceEnabled)
                           dump("Got connection from pool " + cl);
                           return cl;
                           }
                          
                           // Track by transaction
                           try
                           {
                           trackByTx.lock(trackByTransaction);
                           }
                           catch (Throwable t)
                           {
                           JBossResourceException.rethrowAsResourceException("Unable to get connection from the pool for tx=" + trackByTransaction, t);
                           }
                           try
                           {
                           // Already got one
                           ConnectionListener cl = (ConnectionListener) trackByTx.get(trackByTransaction);
                           if (cl != null)
                           {
                           if (traceEnabled)
                           dump("Previous connection tracked by transaction " + cl + " tx=" + trackByTransaction);
                           return cl;
                           }
                           }
                           finally
                           {
                           trackByTx.unlock(trackByTransaction);
                           }
                          
                           // Need a new one for this transaction
                           // This must be done outside the tx local lock, otherwise
                           // the tx timeout won't work and get connection can do a lot of other work
                           // with many opportunities for deadlocks.
                           // Instead we do a double check after we got the transaction to see
                           // whether another thread beat us to the punch.
                           ConnectionListener cl = mcp.getConnection(subject, cri);
                           if (traceEnabled)
                           dump("Got connection from pool tracked by transaction " + cl + " tx=" + trackByTransaction);
                          
                           // Relock and check/set status
                           try
                           {
                           trackByTx.lock(trackByTransaction);
                           }
                           catch (Throwable t)
                           {
                           mcp.returnConnection(cl, false);
                           if (traceEnabled)
                           dump("Had to return connection tracked by transaction " + cl + " tx=" + trackByTransaction + " error=" + t.getMessage());
                           JBossResourceException.rethrowAsResourceException("Unable to get connection from the pool for tx=" + trackByTransaction, t);
                           }
                           try
                           {
                           // Check we weren't racing with another transaction
                           ConnectionListener other = (ConnectionListener) trackByTx.get(trackByTransaction);
                           if (other != null)
                           {
                           mcp.returnConnection(cl, false);
                           if (traceEnabled)
                           dump("Another thread already got a connection tracked by transaction " + other + " tx=" + trackByTransaction);
                           return other;
                           }
                          
                           // This is the connection for this transaction
                           cl.setTrackByTx(true);
                           trackByTx.set(cl);
                           if (traceEnabled)
                           dump("Using connection from pool tracked by transaction " + cl + " tx=" + trackByTransaction);
                           return cl;
                           }
                           finally
                           {
                           trackByTx.unlock(trackByTransaction);
                           }
                           }
                          

                          If you enabled track-connection-by-tx and you are getting the connections in applicaition in a transaction then the underlying connection should be same .

                          You can try this code
                          InitialContext context = new InitialContext();
                           DataSource ds = (DataSource)context.lookup("java:XAOracleDS");
                           //UserTransaction utx = (UserTransaction)context.lookup("UserTransaction");
                           //utx.begin();
                           Connection con = ds.getConnection();
                           Connection con1 = ds.getConnection();
                           WrappedConnection wrap = (WrappedConnection)con;
                           WrappedConnection wrap1 = (WrappedConnection)con1;
                           out.println("UnderlyingConnection is :---> "+wrap.getUnderlyingConnection()+"<br>");
                           out.println("Underlying Connection "+wrap1.getUnderlyingConnection()+"<br>");
                           //utx.commit();
                           con.close();
                           con1.close();
                          

                          In the above code the underlying connections would be different because the transaction related code is commented , if you uncomment that part it will give you same underlying connection.

                          So make sure that the part of code which is giving you different result is in transaction .



                          • 10. Re: Opening connection from a marked rollback tx
                            Vicky Kak Master

                            Huh ... We have deviated from the original problem :)

                            • 11. Re: Opening connection from a marked rollback tx
                              Novice

                              Yes, that's what I expected (that connections are stored in the tx). Hibernate (EJB3) is the one requesting the connection. I am not doing this explicitly there. I might open a connection from the same DS explicitly in another place (within this tx) and then call close() when done. But that should be fine I guess. Of course all work is done in one big tx. It is opened via a UserTransaction from a BMT session bean. The problem occurs before the call returns to the bean that started the tx.

                              Maybe there is a bug in an interceptor, so the tx is lost in the TransactionLocale?

                              Anyway, yes we are drifting away from the problem. I still think it is perfectly legal to open as many connections as I like even if the current tx is set rollback-only. I guess the rollback-only flag is only set in the transaction manager and the connection is completely unaware of this fact until the transaction manager performs the rollback at the end.

                              • 12. Re: Opening connection from a marked rollback tx
                                Vicky Kak Master

                                 

                                "oglueck" wrote:

                                Anyway, yes we are drifting away from the problem. I still think it is perfectly legal to open as many connections as I like even if the current tx is set rollback-only. I guess the rollback-only flag is only set in the transaction manager and the connection is completely unaware of this fact until the transaction manager performs the rollback at the end.

                                When the getConnection() is called on the ConnectionFactory for the xa-tx-datasource the AS should make sure that the associated XA Resource is enlisted in the associated Transaction .
                                Now if you have the following sequence
                                1) Start TX T1
                                2) CF.getConnection(); The associated XAResource with the connection will get enlisted in the T1.
                                3) SET ROLLBACK on T1 . This step basically sets
                                case Status.STATUS_PREPARED:
                                 status = Status.STATUS_MARKED_ROLLBACK;

                                Look at
                                http://anonsvn.jboss.org/repos/jbossas/branches/Branch_4_0/transaction/src/main/org/jboss/tm/TransactionImpl.java
                                4) Again calling the CF.getConnection() will be able to get the same ManagedConnection from the TransactionLocal but when the associated XAResource is getting enlisted in associated TX this code will get called
                                // Inactive transaction
                                 Transaction threadTx = tm.getTransaction();
                                 if (threadTx == null || status != Status.STATUS_ACTIVE)
                                 {
                                 String error = "Transaction " + threadTx + " is not active " + TxUtils.getStatusAsString(status);
                                 if (trace)
                                 log.trace(error + " cl=" + this);
                                 throw new IllegalStateException(error);
                                 }

                                http://anonsvn.jboss.org/repos/jbossas/branches/Branch_4_0/connector/src/main/org/jboss/resource/connectionmanager/TxConnectionManager.java

                                So Connection would not be taken and you would get IllegalStateException .....
                                Enable to TRACE logging on the org.jboss.resource(JBoss JCA) and org.jboss.tm(Transaction Manager) and figure out what is going on ....

                                • 13. Re: Opening connection from a marked rollback tx
                                  Novice

                                  Thanks. I have already had a look at the TM code and saw that check. Trace logging is no option for me. This happens randomly and rarely on a production system.

                                  • 14. Re: Opening connection from a marked rollback tx
                                    Vicky Kak Master

                                     

                                    "oglueck" wrote:
                                    This happens randomly and rarely on a production system.

                                    It then becomes difficult to identify the problem , the only option I can think right now is to simulate the test on the development system .
                                    If you are able to do it there with the TRACE logging enabled for JCA and TM it would be easy to SPOT the cause.


                                    1 2 Previous Next