5 Replies Latest reply on Jul 22, 2013 6:57 AM by Andrew Dinn

    Running Eclipse with byteman javaagent needs immense amount of memory

    Sven Lange Newbie

      Hi!

       

      I wanted to analyse an eclipse plugins behavior with byteman. but when i open an editor i see lots of

       

      Svens-MacBook-Pro:Applications lange$ java -jar eclipse/plugins/org.eclipse.equinox.launcher_1.3.0.v20130327-1440.jar -XstartOnFirstThread -javaagent:${BYTEMAN_HOME}/lib/byteman.jar=script:~/Desktop/blah.btm -XX:MaxPermSize=1500m -Xms3000m -Xmx5000m

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

      java.lang.OutOfMemoryError: PermGen space

       

      error messages on the console.

       

      My system only has 8GB mem and i cannot add additional mem to the eclipse process.

       

      Is byteman soooooo mem hungry??? My rules file has about 8 rules.

        • 1. Re: Running Eclipse with byteman javaagent needs immense amount of memory
          Andrew Dinn Master

          Hi Sven,

          Sven Lange wrote:

           

          . . .

           

          Is byteman soooooo mem hungry??? My rules file has about 8 rules.

          No, Byteman is soooooo *not* memory hungry. It works very hard to avoid retaining references which might use up memory, especially PermGen space. Normally, if you unload all your rules then Byteman will drop all references to any classes they were injected into.

           

          This looks like it is a problem related to application redeployment. This is the most common cause of PermGen exhaustion -- especially inside Eclipse. The way your rules are written may well be causing references to application objects and, hence, classes in perm gen space to be retained across redeployments. If, for example, your rules use application instances as identifiers for Helper objects like counters, flags etc then these objects will be referenced from  static fields (hashtables, ect) declared by Byteman's Helper class and that means their classes will contiue to be retained even after the rules whcih referred to them have been unloaded. Ideally a rule set should always delete such objects at some suitable point -- e.g. every rule set which uses createCounter or incrementCounter should, ideally, include a rule which calls deleteCounter. Of course, this is  normally only a problem in environments like Eclipse where you redeploy classes -- PermGen space is rarely able to be reclaimed. Nor is this a problem unique to Byteman -- many other Java applications/tools fail with PermGen exhaustion

           

          [aside: there are several options available to force dropping of such Helper data whenever all rules whcih might have created them are  unloaded. None of them have  been implemented because they cause issues with retainign backwards comaptibility for existing Byteman rule sets. if your problem turns out to be to do with unwanted retention of application instances thanks to helper calls then I wil be very interested in discussing options for enabling one of other of these forcing mechanisms based on your requirements.]

           

          Another possibility is that you might have found a problem in the retransformer code in your JVM (I assume from your prompt that you are running on OSX). I have little experience of how well OSX implements the agent APIs but it has very likely been tested far less well than Linux. And, of course, you may just have found a nasty memory leak in Byteman itself. The fact that the problem is to do with PermGen space suggests this is to do with reloading classes rather than allocating instances so my expectation is that the problem is something to do with how Byteman is interacting with Eclipse or the JVM. If you could post your rules (or at least give an indication of what they are doing) and explain how you are running your application it would be much easier to identify what is going wrong and work out how to fix it.

          • 2. Re: Running Eclipse with byteman javaagent needs immense amount of memory
            Sven Lange Newbie

            Hi Andrew,

             

            here is my rules script:

             

            RULE trace main entry

            CLASS XtendCompiler

            METHOD getFavoriteVariableName

            AT ENTRY

            IF true

            DO traceln("entering getFavoriteVariableName")

            ENDRULE

             

            RULE trace main exit

            CLASS XtendCompiler

            METHOD normalizeBlockExpression

            AT ENTRY

            IF true

            DO traceln("entering normalizeBlockExpression")

            ENDRULE

             

            RULE trace main exit

            CLASS XtendCompiler

            METHOD doInternalToJavaStatement

            AT ENTRY

            IF true

            DO traceln("entering doInternalToJavaStatement")

            ENDRULE

             

            RULE trace main exit

            CLASS XtendCompiler

            METHOD _toJavaStatement

            AT ENTRY

            IF true

            DO traceln("entering _toJavaStatement")

            ENDRULE

             

            RULE trace main exit

            CLASS XtendCompiler

            METHOD internalToConvertedExpression

            AT ENTRY

            IF true

            DO traceln("entering internalToConvertedExpression")

            ENDRULE

             

            RULE trace main exit

            CLASS XtendCompiler

            METHOD appendCatchClauseParameter

            AT ENTRY

            IF true

            DO traceln("entering appendCatchClauseParameter")

            ENDRULE

             

            RULE trace main exit

            CLASS XtendCompiler

            METHOD appendVariableTypeAndName

            AT ENTRY

            IF true

            DO traceln("entering appendVariableTypeAndName")

            ==== END

             

            Eclipse is running normally until i open a Xtend (http://www.eclipse.org/xtend/) file with the Eclipse Xtend editor view. I can open Java or any other file exceot for Xtend files. As soon as I open a Xtend file I see the error messages on the console and eclipse freezes.

             

            I am using Java 1.6.51 on a macbook with osx 10.4.3 / Eclipse 4.3 Kepler with Xtend 2.4.2

             

            Best

            Sven

            • 3. Re: Running Eclipse with byteman javaagent needs immense amount of memory
              Andrew Dinn Master

              Hi Sven,

              Sven Lange wrote:

               

              here is my rules script:

               

              . . .

               

              RULE trace main exit

              CLASS XtendCompiler

              METHOD appendVariableTypeAndName

              AT ENTRY

              IF true

              DO traceln("entering appendVariableTypeAndName")

              ==== END

               

              Hmm, where it says ==== END does your rule actually say  ENDRULE?

               

              Sven Lange wrote:


              Eclipse is running normally until i open a Xtend (http://www.eclipse.org/xtend/) file with the Eclipse Xtend editor view. I can open Java or any other file exceot for Xtend files. As soon as I open a Xtend file I see the error messages on the console and eclipse freezes.

               

              I am using Java 1.6.51 on a macbook with osx 10.4.3 / Eclipse 4.3 Kepler with Xtend 2.4.2

               

              Well, that is pretty weird. However, I think this is likely to be Eclipse fighting with Byteman in order to retain ownership of what Eclipse almost certainly considers to be one of its own classes.

               

              Byteman uses the JVMTI Java agent capability of the JVM to modify the definition of the classes mentiond in Byteman rules. I don't know for sure what Eclipse does when running Java code but I do know that it will at some point use the JVMTI native library agent capabilities to perform debugging -- that's so because the debugger interface employed by Eclipse and any other program capable of debugging a JVM can only be based on these JVMTI native library APIs (this has been true since at least  JDK6 -- I am not sure but possibly for JDK5 too). So, Eclipse will be using these JVMTI hooks, either directly or indirectly.

               

              JVMTI native library agents can do far more than a Java agent like Byteman. Java agents normally only modify bytecode or, at first load only, Java class field, method and super hierarchy layout. Native agents can modify the behaviour of the JVM as well as redefine Java classes. This means it is hard to be sure exactly what Eclipse is doing without a thorough debug/audit of the Eclipse code itself. However, I suspect Eclipse will want to ensure that its own classes are either i) exempt from being transformed by Java agents or  ii) redefined only in specific classloader contexts so that there is a vanilla version available for Eclipse itself to use. Given the symptoms you describe it looks to me like when Byteman changes the definition of class XtendCompiler that Eclipse is stepping in to provide another version and that this is ending up in some sort of recursive redefinition process.

               

              Can you confirm that you are able to inect rules into other classes which are not related to Eclipse itself? (for example, try injecting code into a JDK class like Thread or String and check that you can still run your program).

               

              Are you forking the code which employs Xtend in a separate JVM from the one running Eclipse? If not then could you investigate if the problem still manifests when you do so?

               

              Thanks for raising this issue and for any further information you can provide. I will raise a JIRA for the problem so that it is documented for other Byteman users. If you can provide more feedback then I might be able to help resolve this problem but I doubt I will be able to do so as I have neither the requisite expertise in Eclipse to debug it properly nor the time to acquire said expertise. Let's hope that someone with more detailed knowledge of Eclipse will be able to idenitfy what is going wrong here and suggest a fix or workaround.

               

              regards,

               

               

              Andrew Dinn

              1 of 1 people found this helpful
              • 4. Re: Running Eclipse with byteman javaagent needs immense amount of memory
                Sven Lange Newbie

                Hi Andrew,

                 

                now i started Eclipse with this rule:

                 

                RULE trace main entry

                CLASS Thread

                METHOD run

                AT ENTRY

                IF true

                DO traceln("entering run")

                ENDRULE

                 

                When I now open a Java editor Eclipse dies immediately without any error message on the console.

                 

                Starting eclipse again it dies again with some errors printed to the console:

                 

                Svens-MacBook-Pro:Applications lange$ java -jar eclipse/plugins/org.eclipse.equinox.launcher_1.3.0.v20130327-1440.jar -XstartOnFirstThread -javaagent:${BYTEMAN_HOME}/lib/byteman.jar=script:~/Desktop/blah.btm -XX:MaxPermSize=1500m -Xms3000m -Xmx5000m

                0    [Worker-1] WARN  org.eclipse.xtext.ui.shared.internal.Activator  - Disabling JDT use. : org/eclipse/jdt/ui/JavaUI

                200  [Worker-1] WARN  org.eclipse.xtext.ui.shared.internal.Activator  - Disabling JDT use. : org/eclipse/jdt/ui/JavaUI

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                An error has occurred. See the log file

                /Users/lange/Desktop/deleteme22/.metadata/.log.

                 

                the log is quite long......

                 

                i wont investigate any further. dont have time to dig deeper into this. thanks for your help!

                • 5. Re: Running Eclipse with byteman javaagent needs immense amount of memory
                  Andrew Dinn Master

                  Hi Sven,

                  Sven Lange wrote:

                   

                  now i started Eclipse with this rule:

                   

                  RULE trace main entry

                  CLASS Thread

                  METHOD run

                  AT ENTRY

                  IF true

                  DO traceln("entering run")

                  ENDRULE

                   

                  When I now open a Java editor Eclipse dies immediately without any error message on the console.

                   

                  Starting eclipse again it dies again with some errors printed to the console:

                   

                  Svens-MacBook-Pro:Applications lange$ java -jar eclipse/plugins/org.eclipse.equinox.launcher_1.3.0.v20130327-1440.jar -XstartOnFirstThread -javaagent:${BYTEMAN_HOME}/lib/byteman.jar=script:~/Desktop/blah.btm -XX:MaxPermSize=1500m -Xms3000m -Xmx5000m

                  0    [Worker-1] WARN  org.eclipse.xtext.ui.shared.internal.Activator  - Disabling JDT use. : org/eclipse/jdt/ui/JavaUI

                  200  [Worker-1] WARN  org.eclipse.xtext.ui.shared.internal.Activator  - Disabling JDT use. : org/eclipse/jdt/ui/JavaUI

                  Job found still running after platform shutdown.  Jobs should be canceled by the plugin that scheduled them during shutdown: org.eclipse.ui.internal.activities.MutableActivityManager$3$1

                  . . .

                   

                  Oh boy, look at all those worms wriggling inside that can!

                   

                  Thanks for following this up. I admit I have no idea what is going on here -- I think this requires an Eclipse GURU :-(

                   

                  Anyway, it's very good to know that there is a problem here.

                   

                  regards,

                   

                   

                  Andrew Dinn