10 Replies Latest reply on Jan 29, 2019 4:29 AM by adinn

    Unable to identify test JVM process during agent load

    simajohn2018

      I am trying to use ByteMan in fault injection tests, but it does not seem to be well maintained at the moment. For example, when I try to use it with JUnit (using BMUnit library), I get the following error:

      Unable to identify test JVM process during agent load

      This issue has been reported some time back and resolved, but surfaced again it seems.

      Has anyone faced this issue recently?

       

      My OS - Windows 10.X.

      Byteman version - 4.0.5.

        • 1. Re: Unable to identify test JVM process during agent load
          adinn

          I am trying to use ByteMan in fault injection tests, but it does not seem to be well maintained at the moment.

           

          That's a rather strange conclusion to draw from the behaviour you are seeing. Byteman is actively maintained and is in regular use by a large number of projects/users. The same applies for BMUnit. Do you mean "but it does not seem to work in my test case on my machine with my configuration"? :-)

           

           

          For example, when I try to use it with JUnit (using BMUnit library), I get the following error:

          Unable to identify test JVM process during agent load

          This issue has been reported some time back and resolved, but surfaced again it seems.

           

           

          Has anyone faced this issue recently?

          My OS - Windows 10.X.

          Byteman version - 4.0.5.

           

           

          Are you referring to issue BYTEMAN-255? That is the only issue I know of that reports the same error message.

           

          If not then could you provide more details?

           

          If so then that issue related to failures in the JVM on Windows 7 (not reporting which JVMs are present on the host machine, including the JVM executing the test code). That is, of course, a problem that Byteman could not fix. Instead Byteman provided a workaround for it based on reading MXBean info. If that workaround is now failing on Windows 10 then the problem is likely to be a different one and most likely indicates, once again, a failure in the JVM or JDK Runtime rather than a failure in Byteman.

           

          The problem may be to do with an OS-specific change in the Windows 10 JVM code, a generic change in the shared JVM code or a change in the JDK Runtime code. Could you please identify which JDK release you are using (I need to see the output when you run command 'java --version'). That will help me understand which version I need to look at in order to identify what is wrong.

           

          regards,

           

           

          Andrew Dinn

          • 2. Re: Unable to identify test JVM process during agent load
            simajohn2018

            It is good to know that it is well maintained, I had big plans of using it for many resiliency tests.

             

            My java version:

            java 10.0.1 2018-04-17

            Java(TM) SE Runtime Environment 18.3 (build 10.0.1+10)

            Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.1+10, mixed

             

            thanks

            • 3. Re: Unable to identify test JVM process during agent load
              adinn

              Hi Simon,

               

              Could you compile and run the following  simple Java program on your machine to check the RutimeMXBean configuration?

               

              import java.lang.management.ManagementFactory;

               

              class RuntimeMXBeanTest
              {
                  public static void main(String[] args) {
                      String processName = ManagementFactory.getRuntimeMXBean().getName();
                      if (processName == null) {
                          processName = "<null>";
                      }
                      System.out.println("id = " + processName);
                  }
              }

               

              If you could tell me what it prints that would be a great help.

              • 4. Re: Unable to identify test JVM process during agent load
                simajohn2018

                sure, the output is:

                 

                id = 10184@CL-SimonR

                • 5. Re: Unable to identify test JVM process during agent load
                  jaikiran

                  Is any IDE being used to run this JUnit test case?

                  • 6. Re: Unable to identify test JVM process during agent load
                    adinn

                    Hi Simon,

                     

                    simajohn2018  wrote:

                     

                    sure, the output is:

                     

                    id = 10184@CL-SimonR

                    Ok, that's very useful info.

                     

                    The MBean property that gets printed out by the test is one set by the JVM when it starts up. That 5 digit number you can at the start of the printed string is the process id of the JVM process. This is relevant because the MBean property is one that the BMUnit code looks for as a last resort when trying to obtain the current process id. The error says I cannto find a process id even using this MBean property as a last resort.

                     

                    I use the phrase "as a last resort" because there is an offical way to obtain the pid that sometimes does not work correctly on Windows. The JVM Tools Interface includes an API (on class VirtualMachine) that BMUnit relies which is supposed to allow the current JVM process to be identified and the queried to obtain it's id. That API started playing up on Windows 7 -- it sometimes fails to access the current JVM for reasons I have not yet been able to pin down. It appears this is still a problem on Windows 10.

                     

                    What is interesting is that the JVM is clearly setting up the necessary MBean property when you run the test program. The question is why is it not present when your tests invoke the Byteman BMUnit code? Can you provide details of how the tests are being run? I noticed that Jaikiran has asked the same question (Hi Jaikiran!). I have a sneaky suspicion he may know what is going on here.

                     

                    It may be that we can resolve this problem simply by stopping whatever it is in your test run config that is i) erasing the MBean property or ii) preventing it from being configured i.e. fix this so the workaround keeps working around. So, any info you can provide about how your tests are being started will be of great help.

                     

                    Meanwhile, I think I need to look deeper into the original problem with class VirtualMachine. As luck would have it, I happen to be an OpenJDK JVM developer as well as the  Byteman project lead :-) Unfortunately, I don't ever build or use it on WIndows, so I only know the relevant JVM code from the Linux side. I will ask some of my OpenJDK team colleagues who do know about the Windows JVM whether they can explain why the VirtualMachine API does not always work.

                     

                    regards,

                     

                     

                    Andrew Dinn

                    • 7. Re: Unable to identify test JVM process during agent load
                      simajohn2018

                      Hi

                       

                      I am running this from intelliJ.

                       

                       

                       

                      Also please see the screenshot of my debugging session. Essentially, Install.getSystemProperty(id, prop); returns null although id has a valid (?) value.

                       

                       

                       

                       

                      Thank you

                       

                      S

                      • 8. Re: Unable to identify test JVM process during agent load
                        adinn

                        Hi Simon,

                         

                        Essentially, Install.getSystemProperty(id, prop); returns null although id has a valid (?) value.

                         

                        Ah, ok, thanks for debugging this. So, this is a new (to me, anyway) limitation in the VirtualMachine API. Before, it would not list active JVMs and read their pids. Now it won't allow you to read a system property from a JVM even when you have the correct pid (ad actually, even when it is the current process).

                         

                        I suspect this might be to do with a hardening up of the JVMTI api that happened in JDK9/10 to control loading of agents into the JVM. One of the things that was scheduled in JDK9 and implemented in JDK10 was the requirement to set a specific property on the command line if you want to be able to load an agent into a running JVM:

                         

                          $ java -Djdk.attach.allowAttachSelf=true . . .

                         

                        Full details are provided in a post I wrote on the Byteman blog shortly after jdk9 came out:

                         

                            Byteman Blog: Configuring BMUnit for the latest JDK9 EA releases

                         

                        To summarise: this property controls the API in VirtuaMachine that allows a JVMTI agent like Byteman to be installed into the current JVM (but, bizarrely, not into some other JVM). If it is false then the BMUnit package cannot load the Byteman agent before running tests. jdk9 noted the setting but did not enforce it. jdk10 and later JVMs enforce it.

                         

                        It may be that this change is also causing these other failures you are seeing on Windows (they won't turn up on Linux because the process id can always be read from the special file in /proc/self/stat). Even if this is not the causeyou will need to ensure that this flag is set on the command line in order to load the agent into jdk10.

                         

                        So, could you try adding it the -D... setting to your test run config and see if it fixes the problem. If not I guess I will have to patch the code to remove the extra test on the 'unique' property.

                         

                        regards,

                         

                         

                        Andrew Dinn

                        • 9. Re: Unable to identify test JVM process during agent load
                          simajohn2018

                          Adding that VM flag solved it. With the flag, the method returns within the first loop which iterates through VMInfo. I hope this is what you expected.

                           

                          thank you very much for the support. Much appreciated!

                           

                           

                           

                          Regards

                           

                          S

                          • 10. Re: Unable to identify test JVM process during agent load
                            adinn

                            Hi Simon,

                             

                            simajohn2018  wrote:

                             

                            Adding that VM flag solved it. With the flag, the method returns within the first loop which iterates through VMInfo. I hope this is what you expected.

                             

                            thank you very much for the support. Much appreciated!

                             

                            Well, that's good to know. Thanks very much for the confirmation.

                             

                            I hope you manage to use Byteman to successfully implement  some of your tests. If you have any other problems getting things to work or you need some help with defining rules for some of your more complex test scenarios please feel free to ask for advice in this forum.

                             

                            regards,

                             

                             

                            Andrew Dinn