7 Replies Latest reply on Aug 22, 2016 9:32 AM by Andrew Dinn

    Using Byteman from API

    Martin Vecera Novice

      Hello,

       

      I would like to use Byteman within my own application to connect the agent to the same JVM running. So far I am copying the behaviour of loadAgent() in https://github.com/bytemanproject/byteman/blob/master/contrib/bmunit/src/org/jboss/byteman/contrib/bmunit/BMUnitConfigState.java

       

      Is there any better way?

       

      I hit the problem that I need JDK's tools.jar as a dependency. Is it still necessary with JDK 8. I think the Java Api was extended to provide the functionality without the tools.jar. Is there a way how to workaround it? Or are there any plans to develop such an extension?

       

      Thanks,

      Martin

        • 1. Re: Using Byteman from API
          Andrew Dinn Master

          Hi Martin,

          Martin Vecera wrote:

           

          Hello,

           

          I would like to use Byteman within my own application to connect the agent to the same JVM running. So far I am copying the behaviour of loadAgent() in https://github.com/bytemanproject/byteman/blob/master/contrib/bmunit/src/org/jboss/byteman/contrib/bmunit/BMUnitConfigState.java

           

          Is there any better way?

           

          I hit the problem that I need JDK's tools.jar as a dependency. Is it still necessary with JDK 8. I think the Java Api was extended to provide the functionality without the tools.jar. Is there a way how to workaround it? Or are there any plans to develop such an extension?

           

          No, there is no better way that I know of to install the agent into the current JVM than the way BMUnit. does it (actually, the heavy lifting is done by class Install but BMUnit provides a way to find the current process).

           

          As far as I am aware the tools jar needs to be in the classpath in JDK8 in order to do attach an agent. Actually, you also need to have byteman-install.jar in your classpath or else replicate the code in install in your app. Install uses the classes in package com.sun.tools.attach and these are the only API I know of which can attach be used to load an agent into a JVM (local or remote).

           

          What made you think the API was changed in JDK8? In JDK9 things have been changed and package com.sun.tools.attach is included in the default JDK image. Perhaps it is that you are thinking of?

          • 2. Re: Using Byteman from API
            Martin Vecera Novice

            Hello Andrew,

             

            thanks for a quick response! I probably confused that with JDK9. I inspected my older code when I attach to the process and I use the same way as in Install. The dependency is quite tricky. I look forward for JDK 9 where this will be much cleaner.

             

            If you were curious, this is where I will be using it - PerfCake/PerfCakeDebug.java at feature/#332-debug-agent · PerfCake/PerfCake · GitHub

             

            Regards!

            Martin

            • 3. Re: Using Byteman from API
              Martin Vecera Novice

              I realized that I cannot use Install. It is unable to find the Byteman jar file properly. It either clashes with my existing installation and its env variable BYTEMAN_HOME, or it does not work from my standalone jar applicationnn that contains classpath in manifest and the system property java.class.path remains empty preventing Install to find the file.

               

              So I worked around this pretty easily and avoided both Install and Submit. My use case might be rare (connecting Byteman agent to my own already running JVM), but you could consider putting this into the documentation:

               

              final String pid = getPid();

              try {

                 final Class clazz = Class.forName("org.jboss.byteman.agent.Main");

                 final VirtualMachine vm = VirtualMachine.attach(pid);

               

                vm.loadAgent(Paths.get(clazz.getProtectionDomain().getCodeSource().getLocation().toURI()).toString(), "listener:false,script:" + saveTmpBytemanRules());

              • 4. Re: Using Byteman from API
                Andrew Dinn Master

                Hi Martin,

                 

                I realized that I cannot use Install. It is unable to find the Byteman jar file properly. It either clashes with my existing installation and its env variable BYTEMAN_HOME, or it does not work from my standalone jar applicationnn that contains classpath in manifest and the system property java.class.path remains empty preventing Install to find the file.

                 

                Yes, you have spotted a nasty problem when it comes to auto-loading the byteman agent jar. If you want to use the code currently employed by class Install then you need to ensure

                 

                either

                • system property org.jboss.byteman.home idenitfies the agent jar that you want to use

                or

                • system property org.jboss.byteman.home is unset and env var BYTEMAN_HOME idenitfies the agent jar that you want to use

                or

                • both vars are unset and a byteman agent jar (/path/to/byteman.jar or /path/to/byteman.x.y.z.jar) is in the classpath.

                 

                Install uses the env var and system property to locate the jar if set. If not then it searches the classpath.

                 

                When running under maven he second requirement means you need to ensure that maven does not use a jar with a manifest to indirectly reference the jars on the classpath. That's a real pain since maven does not really co-operate very well to do that.

                 

                So I worked around this pretty easily and avoided both Install and Submit. My use case might be rare (connecting Byteman agent to my own already running JVM), but you could consider putting this into the documentation:

                 

                final String pid = getPid();

                try {

                  final Class clazz = Class.forName("org.jboss.byteman.agent.Main");

                  final VirtualMachine vm = VirtualMachine.attach(pid);

                  . . .

                vm.loadagent(Paths.get(clazz.getProtectionDomain().getCodeSource().getLocation().toURI()).toString(),

                             "listener.false,script:" + saveTmpBytemanRules());

                 

                Hmm, that's very interesting. I was not aware that you could find the location of a class via the protection domain. This is probably a better way to identify the location of the Byteman jar than searching the class path. I may want to modify the Byteman Install class to use this trick. I'd better be sure it works the same on a Jigsaw-enabled JDK9 though (from a quick eyeball of the code it certainly looks like it does). Thanks very much for posting this!

                 

                Of course, it is critical that you have loaded class org.jboss.byteman.agent.Main to identify the protection domain and no other class. I am assuming you are aware that loading anything else risks linkage problems should you want to hoist the byteman jar into the bootstrap path.

                • 5. Re: Using Byteman from API
                  Andrew Dinn Master

                  Hi Martin,

                   

                  If you were curious, this is where I will be using it - PerfCake/PerfCakeDebug.java at feature/#332-debug-agent · PerfCake/PerfCake · GitHub

                   

                  I am indeed very curious about this. It looks like you are using a fixed set of Byteman rules which you install when the agent is loaded. It looks like that is all you do? Or do you also use class Submit to add extra rules while running?

                   

                  You might want to think about using the javaagent option manager:classname to manage upload of your rules. This option can be used as an alternative to providing startup scripts using script:/path/to/myscript.btm option and can also be used in addition to that option. If you use this to supplly a manager class name then the agent will create an instance of the named class and provide it with a handle on the Retransformer instance that is used to perform injection by calling static method initialize(Retransformer) -- or static method initialize(Retransformer, String, int) if you have provided a String hostname and/or int port number as javaagent options.

                   

                  This means your manager class can call methods to upload and/or unload rules as text rather than having to obtain them from a file. It can also check the state of loaded rules to ensure that they have been injected correctly. Basically it can do whatever is allowed by the public API of Retransformer.

                   

                  This javaagent option has been available for some time now. In fact, when you specify listener:true it is the same as specifying manager:org.jboss.byteman.agent.TransformListener i.e. the default manager is TransformListener which opens a localhost socket allowing class Submit to be used to talk to the agent. If you specify listener:false it results in the manager class name being set to null which means scripts provide don the command line get loaded and no further retransformation is possible.

                  • 6. Re: Using Byteman from API
                    Martin Vecera Novice

                    My usage is very simple. I really do not need to update the rules once they are loaded. I also do not support any manipulation with the bootstrap path. I was really looking for the simplest solution of putting a few rules into my current JVM. The purpose is to publish internal state of PerfCake via JMX. Later these data will be used in an IDE plugin to debug performance test scenario.

                     

                    It is good to know about the manager:classname option. For now, I stick with the current solution as it works. But I might update it in the future. I cannot be sure I will always be able to create a tmp file.

                     

                    If I remember correctly, it was not sufficient to set org.jboss.byteman.home to point to the Byteman jar file while I had the BYTEMAN_HOME set in env pointing to a non existing installation. I would appreciate if the API would be providing less help with localizing the classes and allowed me to set it on my own. Something like Install.agent(pid, jarFile) and it would skip any automatic discoveries. However, this is just a tiny issue and I was able to make it work smoothly.

                     

                    I must say that Byteman is really great for the purpose. Without the need to change a line of code, I am able to expose the complete internal state for debugging.

                    • 7. Re: Using Byteman from API
                      Andrew Dinn Master

                      Ok, if that is all you need then you won't gain much by providing a manager class (just a bit less messing around with the file system).

                       

                      I'm not sure why you observed your BYTEMAN_HOME env setting overriding the setting for property org.jboss.byteman.home. The code in Install definitely checks and uses the system property setting before it will look at the env setting. It passes the first one it finds is set to method locateAgentFromHomeDir(String bmHome).

                       

                      I did notice that there is an error in the way method locateAgentFromHomeDir reports problems with the lookup. It always say that env var BYTEMAN_HOME is invalid even though it might actually be trying to lookup a value passed using property org.jboss.byteman.home. The messages ought to ignore how the value of bmHome is passed and instead just tell you actually that the resulting path is wrong etc. I raised BYTEMAN-327 for that.

                       

                      I'll have a think about how to modify Install to allow the jar to be provided up front. I need to ensure this doesn't break existing code.

                       

                      regards,

                       

                       

                      Andrew Dinn