4 Replies Latest reply on Jun 26, 2015 4:54 AM by jonathan-macke

    Any plan to support annotations in Byteman?

    jonathan-macke

      Hello,

       

      I'm currently testing Byteman for monitoring some key elements of my applications. Very good product. I like it:)

       

      I would like to know if you plan to support annotations in the future. The idea is to instrument methods of classes which are annotated with a given annotation. Annotation will be on the class.

       

      If there is no plan, i will do the evolution on my side and will propose it to you.

       

      Best regards,

       

      Jonathan

        • 1. Re: Any plan to support annotations in Byteman?
          adinn

          Hi Jonathan,

           

          I would like to know if you plan to support annotations in the future. The idea is to instrument methods of classes which are annotated with a given annotation. Annotation will be on the class.

           

          You may already be aware that Byteman has annotation-based instrumentation in the form of the BMUnit package. It only works in conjunction with JUnit or TestNG so is only appropriate for testing rather than monitoring. I mention it for two reasons: in case you have not seen it; and also because it does not work using annotations on the instrumented class (I'lll come to that in a second). If you are not familiar with BMUnit I recommend you take a look at the second and third byteman tutorials for explanations of, respectively, the basic use of annotations and then how to use it to implement complex test scenarios.

           

          Now, as regards the use of annotations, your suggestion that the annotation should be on the class which needs instrumenting was considered when BMUnit was designed. However, it was rejected for several reasons. The most significant problem was that using annotations on the target class means you have to recompile and redeploy the class in order for Byteman to be able to perform instrumentation. With this model the advantages of Byteman are significantly limited -- for most simple scenarios, such as say tracing execution to debug a problem, you could just as easily branch the original source code and add the modifications by hand before rebuilding and redeploying.

           

          With the current approach Byteman allows you to inject into already packaged and deployed classes without any requirement to modify and rebuild code. You can do that when you start a run by providing an agent script at JVM startup. You cna also do that with a running program by uploading the agent into the JVM with the agent listener running and then loading a script into the running program.

           

          The above problem gets worse in circumstances where you have 3rd party code which you are often not in a position to easily modify and rebuild and redeploy. That's often the case for library jars (e.g. stuff you download from maven). It becomes even more tricky if you want to inject into JVM runtime code. You don't want to have to rebuild a whole JDK in order to be able to monitor execution of methods of class FileOutputstream.

           

          Of course, there may be other ways to skin the annotation cat or indeed to improve the management of injection and uninjection by other means than annotations. So, if you have some specific annotation model in mind which you think will bypass these concerns or even if you just want to detail some limitation of the existing behaviour that you are unhappy with please let me know and I will see if there is something I can do to upgrade Byteman.

           

          regards,

           

           

          Andrew Dinn

          • 2. Re: Any plan to support annotations in Byteman?
            jonathan-macke

            Hi Andrew,

             

            Thanks for your reply.

             

            Actually I would like to monitor JPA entities (classical POJO annotated with @Entity).

            For each request on my server (HTTP request, web service request), I would like to detect which fields of my entities are accessed in read and write mode. The idea is to correlate those accessed fields with the SQL queries generated by Hibernate in order to see if we are retrieving the only necessary columns.  We have regular performance issues due to the fact that the SQL generated queries are returning too many columns.

             

            So the idea is to optimize the sql queries by warning the developer.

            In order to do that, I wanted to instrument getter and setter of each JPA entity (I assume that those methods are available in each JPA entity).

             

            If you think there is another way to implement such feature, it will be very helpful for me if you can give me your advice.

             

            Best regards,

             

            Jonathan

            • 3. Re: Any plan to support annotations in Byteman?
              adinn

              Hi Jonathan,

              Actually I would like to monitor JPA entities (classical POJO annotated with @Entity).

               

              Ah, ok. So, it sounds like what you are really interested in is injecting rules into methods of classes which are already annotated but with annotations that are not specific to Byteman. So, something like

               

              RULE foo getter
              CLASS @Entity
              METHOD getFoo()
              . . .
              

              or

               

              RULE foo setter
              CLASS @Entity
              METHOD setFoo(String)
              . . .
              

               

              This is tricky for several reasons both of which boil down to the fact that you are trying to use Byteman to do bulk transformations based on characteristics of classes/methods -- a task it is not really built to do -- rather than to inject into a known method in a specific class or class hierarchy -- the latter being what Byteman actually does with a simple to use model and relatively little runtime overhead. The former is what fairly heavyweight tools like AOP try to do and often do very badly both because of the complexity of the programming model and the inefficiency of the implementation.

               

              So, first I'll explain where the above type of model causes issues for Byteman. Then I'll suggest a way you might be able to do something to help Byteman work round those problems.

               

              Checking for classes with annotations is going to be slow. When a rule is first loaded the whole of the current loaded class base will need to be checked in order to determine if the rule applies to any existing class. That may well involve checking hundreds or, more likely, thousands of classes. Ouch!

               

              After that every time a new class gets loaded the new class's bytecode will need to be checked by Byteman to see if the class has a class-level annotation and then a further check will have to be made to see if any rule mentions that class annotation by name. The latter can just be a simple hash table lookup but the bytecode parsing is going to be slow. Double ouch!

               

              It's a good thing you didn't want to inject rules based on method annotations . . . that takes even more time parsing the byetcode

               

              Compare the above requirement with the normal case where Byteman just has to has the new class's name and see if the hash table contains any rules mentioning that class by name.

               

              Ok, so maybe speed doesn't really matter that much. The second problem is how to specify all the METHODS you are interested in like getName, getAddress etc and also, for setters) specify the associated data types. What constitutes a getter or setter? Well, it has name getXXX or setXXX(TTT xxx) where the @Entity class has a corresponding field with name XXX and type T. Oh dear, so now what you actually need to do is inject into methods based on the name having some pattern based relationship with the attributes of the class. Hmm, Byteman just doesn't provide that sort of capability. Quadruple ouch!

               

              So, how to work around this? Well, you might take a look at the Byteman contrib package called dunit. It's in the Byteman source tree in directory contrib/dunit. It's probably not actually what you need but the basic idea is relevant. What dunit does is use Byteman instrument classes in a remote JVM so that they send trace back to the local JVM. You give dunit a class (or a package) and it uses reflection to idenitfy methods of the class and generate instrumentation rules for those methods (or all classes in the package) then uploads them into the remote JVM. These rules all mention a specific class and a specific method and the rule simply traces method entry.

               

              Now what I suggest you do is adopt a similar technique to generate the rules you need. Write a small standalone app which scans a jar (or jars) containing all the @Entity classes you are interested in. Have your app load each class (you will need to make sure any classes it references are in the classpath) and then use reflection to identify the classes and methods you are interested in. You won't have to worry about uploading rules into a remote JVM. Just write a rule to a script for each setter or getter you want to monitor. Once you have all the rules you need run your program with the Byteman agent and the generated rules installed.

               

              regards,

               

              Andrew Dinn

              • 4. Re: Any plan to support annotations in Byteman?
                jonathan-macke

                Hi Andrew,

                 

                Many thanks for your answer.

                I will give a try by scanning the entities when my app starts (obviously it will have a perf impact but it's not really an issue) and generates the scripts for each getter and setter.

                 

                Best regards,

                 

                Jonathan