6 Replies Latest reply on Aug 24, 2011 6:11 AM by Fekete Kamosh

    Getting stacktrace for timeout transactions

    Fekete Kamosh Newbie

      Hi Arjuna guys,

       

      I found that there are sometimes warnings in my server log (JBossAS [6.0.0.Final "Neo"]) like this:

       

      ARJUNA-12117 TransactionReaper::check timeout for TX 0:ffffc0a80102:126a:4e457d9b:c in state  RUN

      ARJUNA-12095 Abort of action id 0:ffffc0a80102:126a:4e457d9b:c invoked while multiple threads active within it.

      ARJUNA-12108 CheckedAction::check - atomic action 0:ffffc0a80102:126a:4e457d9b:c aborting with 1 threads active!

      ARJUNA-12121 TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,jboss] successfully canceled TX 0:ffffc0a80102:126a:4e457d9b:c

       

      I understand that TransactionReaper checks transaction timeout of enterprise beans that are launched in transaction context due to setting annotation @javax.ejb.TransactionAttribute(TransactionAttributeType.REQUIRED). After timeout rollback is automatically performed. OK.

       

      Anyway I miss information what methods caused timeout and are being rollbacked so that I know where I should maybe increase timeout using @org.jboss.ejb3.annotation.TransactionTimeout annotation.

       

      I found a nice inspiring solution (thank you John Mazzitelli) using interceptor http://svn.rhq-project.org/repos/rhq/trunk/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/common/TransactionInterruptInterceptor.java

      http://community.jboss.org/message/365544#365544

      http://management-platform.blogspot.com/2008/11/transaction-timeouts-and-ejb3jpa.html

       

      But this solution expects public modifier for method com.arjuna.ats.arjuna.coordinator.BasicAction.setCheckedAction(CheckedAction c).

       

      In fact Arjuna implementation changed slightly since 2008 when this solution was published and the method in version of Arjuna (JBOSSTS_4_14_0_Final), which is included in JBossAS 6.0.0, looks like:

       

      com.arjuna.ats.arjuna.coordinator.BasicAction {

         protected final synchronized void setCheckedAction (CheckedAction c)

      }

       

      I vote for encapsalution in general but in this case it is cumbersome for me:-(

       

      Enclosing solution I would like to use, please notice setCheckedAction method which is only workaround of protected modifier I vote against.

       

      package cz.test;
      
      
      import java.io.PrintStream;
      import java.lang.reflect.Method;
      import java.util.Hashtable;
      import javax.interceptor.AroundInvoke;
      import javax.interceptor.InvocationContext;
      import com.arjuna.ats.arjuna.common.Uid;
      import com.arjuna.ats.arjuna.coordinator.BasicAction;
      import com.arjuna.ats.arjuna.coordinator.CheckedAction;
      
      
      /**
       * Interceptor for checking rollbacked transactions. 
       * Inspired by John Mazzitelli
       * http://svn.rhq-project.org/repos/rhq/trunk/modules/enterprise/server/jar/src/main/java/org/rhq/enterprise/server/common/TransactionInterruptInterceptor.java
       */
      public class MyCheckedActionInterceptor {
      
      
                @AroundInvoke
                public Object intercept(InvocationContext ctx) throws Exception {
                          BasicAction currentTx = null;
              try {
                  currentTx = BasicAction.Current();
                  if (currentTx != null) {
                            setCheckedAction(currentTx, new MyCheckedAction());                
                  }
              } catch (Throwable t) {
                  t.printStackTrace();
              }                             
                          return ctx.proceed(); 
                } 
      
      
                private static class MyCheckedAction extends com.arjuna.ats.arjuna.coordinator.CheckedAction {
                          public void check(boolean isCommit, Uid actUid, Hashtable list) {
                                    super.check(isCommit, actUid, list); 
                                    // Show stack trace
                                    for (Object thread : list.values()) {
                                              if (thread instanceof Thread) {
                                                        printStackTrace(System.err, ((Thread) thread).getStackTrace());
                                              }
                                    }
                          }
      
      
                          private void printStackTrace(PrintStream s, StackTraceElement[] stackTrace) {
                                    synchronized (s) {
                                              s.println(this + " stack trace:");
                                              StackTraceElement[] trace = stackTrace;
                                              for (int i = 0; i < trace.length; i++) {
                                                        if (trace[i].getClassName().startsWith("cz.test")) {
                                                                  s.println("\tat " + trace[i]);
                                                        }
                                              }
                                    }
                          }
                }
      
                // I do not like this workaround to bypass protected modifier
                private void setCheckedAction(BasicAction basicAction, CheckedAction checkedAction) { 
                          try {
                                    Method setCheckedActionMethod = BasicAction.class.getDeclaredMethod("setCheckedAction", new Class[]{CheckedAction.class});
                                    setCheckedActionMethod.setAccessible(true);
                                    setCheckedActionMethod.invoke(basicAction, checkedAction);
                          } catch (Exception e) {
                                    e.printStackTrace();
                          }
                }
      }
      

       

       

      If you have any other solution how to get stacktrace of timeout rollbacked transaction, please give me a clue.

       

      Thank you

       

      Fekete Kamosh

        • 1. Re: Getting stacktrace for timeout transactions
          Jonathan Halliday Master

          I don't have the codebase in front of me at present, but IIRC you should install the handler via a factory, so you need something along the lines of

           

          <code>

          public class MyCheckedActionFactory implements CheckedActionFactory {

            public CheckedAction getCheckedAction (final Uid txId, final String actionType) {

              return new MyCheckedAction(...);

            }

          }

           

          arjPropertyManager.getCoordinatorEnvironmentBean().setCheckedActionFactory(MyCheckedActionFactory)

          </code>

          • 2. Re: Getting stacktrace for timeout transactions
            Fekete Kamosh Newbie

            Hi,

            I am afraid that setting of arjPropertyManager and its own checked action factory does not help because BasicAction itself already holds

             

            private static CheckedActionFactory _checkedActionFactory = arjPropertyManager.getCoordinatorEnvironmentBean().getCheckedActionFactory();
            

             

            The only solution I have found is to add new setting to transaction-jboss-beans.xml for bean CoordinatorEnvironmentBean and set there own factory class

             

            <property name="checkedActionFactoryClassName">cz.test.MyCheckedActionFactory</property>
            

             

             

            But I would like to setup this setting in runtime. And setting in runtime leads me to another ugly solution to bypass private modifier of _checkedActionFactory

             

            try {
                    Field checkedActionFactoryField = BasicAction.class.getDeclaredField("_checkedActionFactory");;
                    checkedActionFactoryField.setAccessible(true);
                    checkedActionFactoryField.set(new BasicAction(), new MyCheckedActionFactoryImple());
            } catch(Exception e) {
                e.printStackTrace();
            }
            

             

            Please reconsider at least private modifier of _checkedActionFactory or maybe better - create setter for this.

             

            Fekete

            • 3. Re: Getting stacktrace for timeout transactions
              Jonathan Halliday Master

              No. Use the xml or just set up the factory at runtime before classloading BasicAction.

              • 4. Re: Getting stacktrace for timeout transactions
                Fekete Kamosh Newbie

                So there is no possibility (and there will no be ever possibility )to setup CheckedAction in runtime and to reset it to original default behaviour?

                 

                I can imagine situations that for some transactions you might want to:

                 

                 

                Fekete

                • 5. Re: Getting stacktrace for timeout transactions
                  Jonathan Halliday Master

                  I'm merely ruling out a short term, ad-hoc solution to this single issue. It's not a good use of our time and we can't promise to support the api going forward. The options already present are adequate for the purpose.  In the longer term I want to systematically rework the configuration system to allow dynamic reconfiguration, running multiple parallel configurations and other nifty stuff. Doing that in a way that won't impact performance or supportability is non trivial and the work will likely span several releases. Indeed the conversion to a bean based property system that has already taken place is the first step on this path. There are over 100 config options in TS and I don't want to wind up with a different solution for each of them, I just want to focus on solving the problem in a clean, uniform way for the whole system.

                  1 of 1 people found this helpful
                  • 6. Re: Getting stacktrace for timeout transactions
                    Fekete Kamosh Newbie

                    OK, in this case I consider this as concluded and I am looking for dynamic runtime reconfiguration in future releases.

                     

                    Fekete