6 Replies Latest reply on Aug 24, 2011 6:11 AM by fekete_kamosh

    Getting stacktrace for timeout transactions

    fekete_kamosh

      Hi Arjuna guys,

       

      I found that there are sometimes warnings in my server log (JBossAS [6.0.0.Final "Neo"]) like this:

       

      ARJUNA-12117 TransactionReaper::check timeout for TX 0:ffffc0a80102:126a:4e457d9b:c in state  RUN

      ARJUNA-12095 Abort of action id 0:ffffc0a80102:126a:4e457d9b:c invoked while multiple threads active within it.

      ARJUNA-12108 CheckedAction::check - atomic action 0:ffffc0a80102:126a:4e457d9b:c aborting with 1 threads active!

      ARJUNA-12121 TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,jboss] successfully canceled TX 0:ffffc0a80102:126a:4e457d9b:c

       

      I understand that TransactionReaper checks transaction timeout of enterprise beans that are launched in transaction context due to setting annotation @javax.ejb.TransactionAttribute(TransactionAttributeType.REQUIRED). After timeout rollback is automatically performed. OK.

       

      Anyway I miss information what methods caused timeout and are being rollbacked so that I know where I should maybe increase timeout using @org.jboss.ejb3.annotation.TransactionTimeout annotation.

       

      I found a nice inspiring solution (thank you John Mazzitelli) using interceptor http://svn.rhq-project.org/repos/rhq/trunk/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/common/TransactionInterruptInterceptor.java

      http://community.jboss.org/message/365544#365544

      http://management-platform.blogspot.com/2008/11/transaction-timeouts-and-ejb3jpa.html

       

      But this solution expects public modifier for method com.arjuna.ats.arjuna.coordinator.BasicAction.setCheckedAction(CheckedAction c).

       

      In fact Arjuna implementation changed slightly since 2008 when this solution was published and the method in version of Arjuna (JBOSSTS_4_14_0_Final), which is included in JBossAS 6.0.0, looks like:

       

      com.arjuna.ats.arjuna.coordinator.BasicAction {

         protected final synchronized void setCheckedAction (CheckedAction c)

      }

       

      I vote for encapsalution in general but in this case it is cumbersome for me:-(

       

      Enclosing solution I would like to use, please notice setCheckedAction method which is only workaround of protected modifier I vote against.

       

      package cz.test;
      
      
      import java.io.PrintStream;
      import java.lang.reflect.Method;
      import java.util.Hashtable;
      import javax.interceptor.AroundInvoke;
      import javax.interceptor.InvocationContext;
      import com.arjuna.ats.arjuna.common.Uid;
      import com.arjuna.ats.arjuna.coordinator.BasicAction;
      import com.arjuna.ats.arjuna.coordinator.CheckedAction;
      
      
      /**
       * Interceptor for checking rollbacked transactions. 
       * Inspired by John Mazzitelli
       * http://svn.rhq-project.org/repos/rhq/trunk/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/common/TransactionInterruptInterceptor.java
       */
      public class MyCheckedActionInterceptor {
      
      
                @AroundInvoke
                public Object intercept(InvocationContext ctx) throws Exception {
                          BasicAction currentTx = null;
              try {
                  currentTx = BasicAction.Current();
                  if (currentTx != null) {
                            setCheckedAction(currentTx, new MyCheckedAction());                
                  }
              } catch (Throwable t) {
                  t.printStackTrace();
              }                             
                          return ctx.proceed(); 
                } 
      
      
                private static class MyCheckedAction extends com.arjuna.ats.arjuna.coordinator.CheckedAction {
                          public void check(boolean isCommit, Uid actUid, Hashtable list) {
                                    super.check(isCommit, actUid, list); 
                                    // Show stack trace
                                    for (Object thread : list.values()) {
                                              if (thread instanceof Thread) {
                                                        printStackTrace(System.err, ((Thread) thread).getStackTrace());
                                              }
                                    }
                          }
      
      
                          private void printStackTrace(PrintStream s, StackTraceElement[] stackTrace) {
                                    synchronized (s) {
                                              s.println(this + " stack trace:");
                                              StackTraceElement[] trace = stackTrace;
                                              for (int i = 0; i < trace.length; i++) {
                                                        if (trace[i].getClassName().startsWith("cz.test")) {
                                                                  s.println("\tat " + trace[i]);
                                                        }
                                              }
                                    }
                          }
                }
      
                // I do not like this workaround to bypass protected modifier
                private void setCheckedAction(BasicAction basicAction, CheckedAction checkedAction) { 
                          try {
                                    Method setCheckedActionMethod = BasicAction.class.getDeclaredMethod("setCheckedAction", new Class[]{CheckedAction.class});
                                    setCheckedActionMethod.setAccessible(true);
                                    setCheckedActionMethod.invoke(basicAction, checkedAction);
                          } catch (Exception e) {
                                    e.printStackTrace();
                          }
                }
      }
      

       

       

      If you have any other solution how to get stacktrace of timeout rollbacked transaction, please give me a clue.

       

      Thank you

       

      Fekete Kamosh

        • 1. Re: Getting stacktrace for timeout transactions
          jhalliday

          I don't have the codebase in front of me at present, but IIRC you should install the handler via a factory, so you need something along the lines of

           

          <code>

          public class MyCheckedActionFactory implements CheckedActionFactory {

            public CheckedAction getCheckedAction (final Uid txId, final String actionType) {

              return new MyCheckedAction(...);

            }

          }

           

          arjPropertyManager.getCoordinatorEnvironmentBean().setCheckedActionFactory(MyCheckedActionFactory)

          </code>

          • 2. Re: Getting stacktrace for timeout transactions
            fekete_kamosh

            Hi,

            I am afraid that setting of arjPropertyManager and its own checked action factory does not help because BasicAction itself already holds

             

            private static CheckedActionFactory _checkedActionFactory = arjPropertyManager.getCoordinatorEnvironmentBean().getCheckedActionFactory();
            

             

            The only solution I have found is to add new setting to transaction-jboss-beans.xml for bean CoordinatorEnvironmentBean and set there own factory class

             

            <property name="checkedActionFactoryClassName">cz.test.MyCheckedActionFactory</property>
            

             

             

            But I would like to setup this setting in runtime. And setting in runtime leads me to another ugly solution to bypass private modifier of _checkedActionFactory

             

            try {
                    Field checkedActionFactoryField = BasicAction.class.getDeclaredField("_checkedActionFactory");;
                    checkedActionFactoryField.setAccessible(true);
                    checkedActionFactoryField.set(new BasicAction(), new MyCheckedActionFactoryImple());
            } catch(Exception e) {
                e.printStackTrace();
            }
            

             

            Please reconsider at least private modifier of _checkedActionFactory or maybe better - create setter for this.

             

            Fekete

            • 3. Re: Getting stacktrace for timeout transactions
              jhalliday

              No. Use the xml or just set up the factory at runtime before classloading BasicAction.

              • 4. Re: Getting stacktrace for timeout transactions
                fekete_kamosh

                So there is no possibility (and there will no be ever possibility )to setup CheckedAction in runtime and to reset it to original default behaviour?

                 

                I can imagine situations that for some transactions you might want to:

                 

                 

                Fekete

                • 5. Re: Getting stacktrace for timeout transactions
                  jhalliday

                  I'm merely ruling out a short term, ad-hoc solution to this single issue. It's not a good use of our time and we can't promise to support the api going forward. The options already present are adequate for the purpose.  In the longer term I want to systematically rework the configuration system to allow dynamic reconfiguration, running multiple parallel configurations and other nifty stuff. Doing that in a way that won't impact performance or supportability is non trivial and the work will likely span several releases. Indeed the conversion to a bean based property system that has already taken place is the first step on this path. There are over 100 config options in TS and I don't want to wind up with a different solution for each of them, I just want to focus on solving the problem in a clean, uniform way for the whole system.

                  1 of 1 people found this helpful
                  • 6. Re: Getting stacktrace for timeout transactions
                    fekete_kamosh

                    OK, in this case I consider this as concluded and I am looking for dynamic runtime reconfiguration in future releases.

                     

                    Fekete