4 Replies Latest reply on Apr 15, 2010 2:01 PM by jaikiran

    Rule execution for java.util.concurrent.ThreadPoolExecutor

    jaikiran

      I am trying out Byteman (1.2.2) against JBoss AS. The rule script includes a rule for java.util.concurrent.ThreadPoolExecutor and one for a JBoss specific class: {code} RULE ThreadPoolExecutor_Creation CLASS ^java.util.concurrent.ThreadPoolExecutor METHOD AT ENTRY IF TRUE DO System.out.println("Creating threadpool executor") ENDRULE RULE ThreadPoolExecutor_Shutdown CLASS ^java.util.concurrent.ThreadPoolExecutor METHOD shutdown AT ENTRY IF TRUE DO System.out.println("Shutting down threadpool executor") ENDRULE RULE test123 CLASS org.jboss.bootstrap.impl.as.server.JBossASServerImpl METHOD AT ENTRY IF TRUE DO System.out.println("hellllooooo") ENDRULE {code} The startup script includes enabling verbose logging and dumping the class file: {code} JAVA_OPTS="-Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -javaagent:$BYTEMAN_HOME/lib/byteman.jar=script:/home/jaikiran/Desktop/bytemanrule.txt -Dorg.jboss.byteman.verbose -Dorg.jboss.byteman.dump.generated.classes -Dorg.jboss.byteman.dump.generated.classes.directory=/home/jaikiran/deleteme" {code} When I start the JBoss AS, I can see that the JVM options are picked up and even verbose logging is triggered: {code} jaikiran@jaikiran-desktop:bin$ ./run.sh -b 0.0.0.0 =========================================================================   JBoss Bootstrap Environment   JBOSS_HOME: /home/jaikiran/jbossas/trunk/build/target/jboss-6.0.0-SNAPSHOT   JAVA: /opt/Java/SunJava-6/jdk1.6.0_16//bin/java   JAVA_OPTS: -server -Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -javaagent:/home/jaikiran/byteman/lib/byteman.jar=script:/home/jaikiran/Desktop/bytemanrule.txt -Dorg.jboss.byteman.transform.all -Dorg.jboss.byteman.verbose -Dorg.jboss.byteman.dump.generated.classes -Dorg.jboss.byteman.dump.generated.classes.directory=/home/jaikiran/deleteme -Djava.net.preferIPv4Stack=true -Dprogram.name=run.sh -Djava.library.path=/home/jaikiran/jbossas/trunk/build/target/jboss-6.0.0-SNAPSHOT/bin/native/lib   CLASSPATH: /home/jaikiran/jbossas/trunk/build/target/jboss-6.0.0-SNAPSHOT/bin/run.jar:/opt/Java/SunJava-6/jdk1.6.0_16//lib/tools.jar ========================================================================= org.jboss.byteman.agent.Transformer : possible trigger for rule test123 in class org.jboss.bootstrap.impl.as.server.JBossASServerImpl RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into method () void for rule test123 RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into method (org.jboss.bootstrap.api.as.config.JBossASServerConfig) void for rule test123 org.jboss.byteman.agent.Transformer : inserted trigger for test123 in class org.jboss.bootstrap.impl.as.server.JBossASServerImpl org.jboss.byteman.agent.Transformer : Saving transformed bytes to /home/jaikiran/deleteme/org/jboss/bootstrap/impl/as/server/JBossASServerImpl.class Rule.execute called for test123_1 test123 execute hellllooooo Rule.execute called for test123_0 test123 execute hellllooooo 22:41:24,074 INFO  [AbstractJBossASServerBase] Server Configuration:     JBOSS_HOME URL: file:/home/jaikiran/NotBackedUp/business/jboss/wc/jbossas/trunk/build/target/jboss-6.0.0-SNAPSHOT/     Bootstrap: $JBOSS_HOME/server/default/conf/bootstrap.xml     Common Base: $JBOSS_HOME/common/     Common Library: $JBOSS_HOME/common/lib/     Server Name: default     Server Base: $JBOSS_HOME/server/     Server Library: $JBOSS_HOME/server/default/lib/     Server Config: $JBOSS_HOME/server/default/conf/     Server Home: $JBOSS_HOME/server/default/     Server Data: $JBOSS_HOME/server/default/data/     Server Log: $JBOSS_HOME/server/default/log/     Server Temp: $JBOSS_HOME/server/default/tmp/ 22:41:24,082 INFO  [AbstractServer] Starting: JBossAS [6.0.0.SNAPSHOT "Neo"] org.jboss.byteman.agent.Transformer : possible trigger for rule ThreadPoolExecutor_Creation in class java.util.concurrent.ThreadPoolExecutor RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into method (int,int,long,java.util.concurrent.TimeUnit,java.util.concurrent.BlockingQueue) void for rule ThreadPoolExecutor_Creation RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into method (int,int,long,java.util.concurrent.TimeUnit,java.util.concurrent.BlockingQueue,java.util.concurrent.ThreadFactory) void for rule ThreadPoolExecutor_Creation RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into method (int,int,long,java.util.concurrent.TimeUnit,java.util.concurrent.BlockingQueue,java.util.concurrent.RejectedExecutionHandler) void for rule ThreadPoolExecutor_Creation RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into method (int,int,long,java.util.concurrent.TimeUnit,java.util.concurrent.BlockingQueue,java.util.concurrent.ThreadFactory,java.util.concurrent.RejectedExecutionHandler) void for rule ThreadPoolExecutor_Creation org.jboss.byteman.agent.Transformer : inserted trigger for ThreadPoolExecutor_Creation in class java.util.concurrent.ThreadPoolExecutor org.jboss.byteman.agent.Transformer : possible trigger for rule log on threadpoolexecutor shutdown in class java.util.concurrent.ThreadPoolExecutor RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into method shutdown() void for rule log on threadpoolexecutor shutdown org.jboss.byteman.agent.Transformer : inserted trigger for log on threadpoolexecutor shutdown in class java.util.concurrent.ThreadPoolExecutor 22:41:27,514 INFO  [ServerInfo] Java version: 1.6.0_16,Sun Microsystems Inc. 22:41:27,515 INFO  [ServerInfo] Java Runtime: Java(TM) SE Runtime Environment (build 1.6.0_16-b01) 22:41:27,515 INFO  [ServerInfo] Java VM: Java HotSpot(TM) Server VM 14.2-b01,Sun Microsystems Inc. 22:41:27,515 INFO  [ServerInfo] OS-System: Linux 2.6.28-18-generic,i386 22:41:27,515 INFO  [ServerInfo] VM arguments: -Xms128m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -javaagent:/home/jaikiran/NotBackedUp/business/me/byteman/lib/byteman.jar=script:/home/jaikiran/Desktop/bytemanrule.txt -Dorg.jboss.byteman.verbose -Dorg.jboss.byteman.dump.generated.classes -Dorg.jboss.byteman.dump.generated.classes.directory=/home/jaikiran/deleteme -Djava.net.preferIPv4Stack=true -Dprogram.name=run.sh -Djava.library.path=/home/jaikiran/NotBackedUp/business/jboss/wc/jbossas/trunk/build/target/jboss-6.0.0-SNAPSHOT/bin/native/lib -Djava.endorsed.dirs=/home/jaikiran/NotBackedUp/business/jboss/wc/jbossas/trunk/build/target/jboss-6.0.0-SNAPSHOT/lib/endorsed 22:41:27,570 INFO  [JMXKernel] Legacy JMX core initialized ... {code} but the rule for java.util.concurrent.ThreadPoolExecutor is never *executed*. The logs just show that the transformation is inserted, but there's no rule execution. Unlike for java.* package, I can see that for the JBoss specific package, the rule is executed: {code} 22:34:59,450 INFO  [STDOUT] org.jboss.byteman.agent.Transformer : possible trigger for rule test456 in class org.jboss.system.server.jmx.JMXConnector 22:34:59,451 INFO  [STDOUT] RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into method () void for rule test456 22:34:59,453 INFO  [STDOUT] org.jboss.byteman.agent.Transformer : inserted trigger for test456 in class org.jboss.system.server.jmx.JMXConnector 22:34:59,454 INFO  [STDOUT] org.jboss.byteman.agent.Transformer : Saving transformed bytes to /home/jaikiran/deleteme/org/jboss/system/server/jmx/JMXConnector.class 22:34:59,465 INFO  [STDOUT] Rule.execute called for test456_7 22:34:59,466 INFO  [STDOUT] test456 execute 22:34:59,466 INFO  [STDOUT] wohooooooooooooooooooooooooooooooooooooooooooooo {code} Also, the transformed class file is dumped only for org.jboss.* class and not for java.* class. Is there something I am missing with enabling rule execution for java.* package? P.S: I have tried adding -Dorg.jboss.byteman.transform.all property, but that didn't help (I did not expect that to work anyway :) )

        • 1. Re: Rule execution for java.util.concurrent.ThreadPoolExecutor
          adinn

          Hi Jaikiran,

           

          You do need to use the property -Dorg.jboss.byteman.transform.all because you are trying to inject into a java.util class and normally java.XXX classes are exempt from transformation. The other problem you are facing is that the target class is in the bootstrap classpath. Normally the agent jar is added to the system classpath and this causes errors when it tries to transform code which is in the bootstrap classpath (basically, some of the types used in injected code are in the byteman jar and these cannot be resolved). In order to fix this you need toadd an extra "boot:<jar>" option to your javaagent command line argument needs:

           

          -javaagent:$BYTEMAN_HOME/lib/byteman.jar=script:/home/jaikiran/Desktop/bytemanrule.txt,boot:$BYTEMAN_HOME/lib/byteman.jar

          This ensures that the agent jar and, hence, agent classes are vsiible when injecting code into bootstrap classes.

           

          The second problem si that your rule for the constructor is incorrect. You need to use a METHDO keyword with  name <init> to refer to a constructor

           

          RULE ThreadPoolExecutor_Creation
          CLASS ^java.util.concurrent.ThreadPoolExecutor
          METHOD <init>
          AT ENTRY
          IF TRUE
          DO System.out.println("Creating threadpool executor " + $0)
          ENDRULE

           

          I know this looks a bit cheesy but <init> is the internal name used in the bytecode. You can also use METHOD <init>(int) or some such variant if you want the match to be specific to the constructor with a given signature. Also, you can use <clinit> to match the class initialiser.

           

          Here is a simple test program, rule script and command line whcih I cooked up to do what you want.

           

          File ThreadExecTest.java

          import java.util.concurrent.ThreadPoolExecutor;
          import java.util.concurrent.ScheduledThreadPoolExecutor;

          public class ThreadExecTest
          {
              public static void main(String[] args)
              {
              ThreadPoolExecutor executor = new ScheduledThreadPoolExecutor(5);
              executor.shutdown();
              }
          }


          File script.txt

           

          RULE ThreadPoolExecutor_Creation
          CLASS ^java.util.concurrent.ThreadPoolExecutor
          METHOD <init>
          AT ENTRY
          IF TRUE
          DO System.out.println("Creating threadpool executor " + $0)
          ENDRULE

          RULE ThreadPoolExecutor_Shutdown
          CLASS ^java.util.concurrent.ThreadPoolExecutor
          METHOD shutdown
          AT ENTRY
          IF TRUE
          DO System.out.println("Shutting down threadpool
          executor " + $0)
          ENDRULE

           

          Command line and output

           

          [adinn@localhost threadexec]$ java -Dorg.jboss.byteman.transform.all -javaagent:$BYTEMAN_HOME/lib/byteman.jar=script:script.txt,boot:$BYTEMAN_HOME/lib/byteman.jar ThreadExecTest
          Creating threadpool executor java.util.concurrent.ScheduledThreadPoolExecutor@139eeda
          Shutting down threadpool executor java.util.concurrent.ScheduledThreadPoolExecutor@139eeda
          [adinn@localhost threadexec]$

           

          I hope that sorts out your problem. Let me know if anything else goes wrong.

           

          regards,

           

           

          Andrew Dinn

          • 2. Re: Rule execution for java.util.concurrent.ThreadPoolExecutor
            jaikiran

            Andrew Dinn wrote:

             

            Hi Jaikiran,

             

            You do need to use the property -Dorg.jboss.byteman.transform.all because you are trying to inject into a java.util class and normally java.XXX classes are exempt from transformation.

            I had initially thought that this might be the issue, but the docs (and even the code) appears to be looking only for java.lang.* package and not java.util.* packages. So I decided to leave that param out (I did try it out once though).

             

            Andrew Dinn wrote:

             

            Hi Jaikiran,

             

            The other problem you are facing is that the target class is in the bootstrap classpath. Normally the agent jar is added to the system classpath and this causes errors when it tries to transform code which is in the bootstrap classpath (basically, some of the types used in injected code are in the byteman jar and these cannot be resolved). In order to fix this you need toadd an extra "boot:<jar>" option to your javaagent command line argument needs:

             

            -javaagent:$BYTEMAN_HOME/lib/byteman.jar=script:/home/jaikiran/Desktop/bytemanrule.txt,boot:$BYTEMAN_HOME/lib/byteman.jar

            This ensures that the agent jar and, hence, agent classes are vsiible when injecting code into bootstrap classes.

             


            Yeah, I missed that part in the documentation. Will give the boot: param a try.

             

            Andrew Dinn wrote:

             


             

            The second problem si that your rule for the constructor is incorrect. You need to use a METHDO keyword with  name <init> to refer to a constructor

             

            RULE ThreadPoolExecutor_Creation
            CLASS ^java.util.concurrent.ThreadPoolExecutor
            METHOD <init>
            AT ENTRY
            IF TRUE
            DO System.out.println("Creating threadpool executor " + $0)
            ENDRULE

             

            Ah, this part is an issue with this (messed up) forum editor. I don't know why it ate up the &lt;init;gt; from the code I posted earlier.

             

            Andrew Dinn wrote:

             


             

            Here is a simple test program, rule script and command line whcih I cooked up to do what you want.

            ...

            Thanks Andrew, I'll be giving this a try today.

             

            By the way, while looking into this, I notice 2 issues one in code and one in documentation.

             

            1) The documentation says setting the org.jboss.byteman.skip.overriding.rules system property. This appears to be a typo, because looking at the code, the real property name is org.jboss.byteman.skip.override.rules. I then set it accordingly in my startup scripts.

             

            2) There appears to be NPE happening (which in part is contributing to some of the issues that I am running into) within the Transformer which results to the transformer skipping the transformation. There's this block:

             

            if (!skipOverrideRules()) {
                            // ok, now check the superclass for this class and so on
            
                            String superName = checker.getSuper();
            
                            while (superName != null) {
                                // we need to check the super class structure
                                // n.b. we use the original loader here because we don't want to search the system loader
                                // when we have a class in the bootstrap loader
                                checker = getClassChecker(superName, originalLoader);
            
                                ...
                    } finally {
                        if (enabled) {
                            Rule.enableTriggersInternal();
                        }
                    }
            

             

            The originalLoader being passed to getClassChecker happens to be null (i.e. bootstrap loader) which results in a NPE deep within getClassChecker and goes uncaught, which ultimately results in the ThreadPoolExecutor going untransformed.

             

            I was able to workaround the NPE by adding the org.jboss.byteman.skip.override.rules property to bypass that piece of code. I'll try out your other suggestions related to adding the jar to boot classpath (infact, I had ended up with the CNFE and NCDFE for byteman classes after I moved one step ahead solving the NPE). I'll let you know how it goes.

            • 3. Re: Rule execution for java.util.concurrent.ThreadPoolExecutor
              adinn

              By the way, while looking into this, I notice 2 issues one in code and one in documentation.

               

              1) The documentation says setting the org.jboss.byteman.skip.overriding.rules system property. This appears to be a typo, because looking at the code, the real property name is org.jboss.byteman.skip.override.rules. I then set it accordingly in my startup scripts.

               

              2) There appears to be NPE happening (which in part is contributing to some of the issues that I am running into) within the Transformer which results to the transformer skipping the transformation. There's this block:

               

               

              Ok, I'll sort out #1 before the netx release

               

              I looked at #2 and I found and fixed the problem (JIRA is https://jira.jboss.org/jira/browse/BYTEMAN-90).

               

              If getClassChecker has not already seen a superclass it tries to load it's bytecode via the supplied loader using baseLoader.getResourceAsStream(). Obviously this fails with an NPE if the loader is null. My test example in the previous note only worked by fluke and only in part -- injection into ScheduledThreadPoolExecutor worked but not into ThreadPoolExecutor. I should have seen several constructor calls -- ScheduledThreadPoolExecutor(int) indirects to ThreadPoolExecutor(int,int,long,TimeUnit,BlockingQueue<Runnable>) which indirects to ThreadPoolExecutor(int,int,long,TimeUnit,BlockingQueue<Runnable>, ThreadFavtory, RejectedExecutionHandler) and also two calls to the shutdown method one in the subclass and one in the parent class.

               

              I have patched my trunk build so that getClassChecker uses the system classloader to do the getResourceAsStream() call if the loader is null and this fixes the problem. I now see the following output

               

              [adinn@localhost threadexec]$ java -Dorg.jboss.byteman.transform.all -javaagent:$BYTEMAN_HOME/lib/byteman.jar=script:script.txt,boot:$BYTEMAN_HOME/lib/byteman.jar ThreadExecTest
              Creating threadpool executor java.util.concurrent.ScheduledThreadPoolExecutor@2a4983
              Creating threadpool executor java.util.concurrent.ScheduledThreadPoolExecutor@2a4983
              Creating threadpool executor java.util.concurrent.ScheduledThreadPoolExecutor@2a4983
              Shutting down threadpool executor java.util.concurrent.ScheduledThreadPoolExecutor@2a4983
              Shutting down threadpool executor java.util.concurrent.ScheduledThreadPoolExecutor@2a4983
              [adinn@localhost threadexec]$

              This shows each constructor call being made and also the calls to shutdown in the subclass and superclass.

              • 4. Re: Rule execution for java.util.concurrent.ThreadPoolExecutor
                jaikiran

                jaikiran pai wrote:

                 


                Andrew Dinn wrote:

                 

                Hi Jaikiran,

                 

                The other problem you are facing is that the target class is in the bootstrap classpath. Normally the agent jar is added to the system classpath and this causes errors when it tries to transform code which is in the bootstrap classpath (basically, some of the types used in injected code are in the byteman jar and these cannot be resolved). In order to fix this you need toadd an extra "boot:<jar>" option to your javaagent command line argument needs:

                 

                -javaagent:$BYTEMAN_HOME/lib/byteman.jar=script:/home/jaikiran/Desktop/bytemanrule.txt,boot:$BYTEMAN_HOME/lib/byteman.jar

                This ensures that the agent jar and, hence, agent classes are vsiible when injecting code into bootstrap classes.

                 


                Yeah, I missed that part in the documentation. Will give the boot: param a try.

                 

                ...
                I'll try out your other suggestions related to adding the jar to boot classpath (infact, I had ended up with the CNFE and NCDFE for byteman classes after I moved one step ahead solving the NPE). I'll let you know how it goes.

                Setting the boot: option got it working. Thanks for all the help!

                 

                 

                Andrew Dinn wrote:

                Ok, I'll sort out #1 before the netx release

                 

                I looked at #2 and I found and fixed the problem (JIRA is https://jira.jboss.org/jira/browse/BYTEMAN-90).

                Thank you.