Byteman rules seem to be silently ignored on jdk11

[bdeggleston]

Hi all,

I'm running into an issue where Byteman scripts work as expected on jdk8, but seem like they're just not getting installed on jdk11. I'm not seeing any exceptions or anything, the rules are just being silently ignored. I am setting `-Djdk.attach.allowAttachSelf=true`

I have a BMUnit test demonstrating this here: BMSanityCheckTest.java · GitHub

On jdk8, the test output is this: jdk8 · GitHub , and you can see the rule being attached and working as expected.

On jdk11, the test output is this: jdk11 · GitHub , clearly a lot less going on in the agent.

I've tried stepping through the agent code in the idea debugger, but the debugger seems to be having a problem with the debug symbols and execution isn't (or doesn't appear to be) following the code. I'm guessing there's some bytecode magic happening that's confusing the debugger, but in any case it's making it difficult to understand what's happening here.

Any insight would be appreciated.

Thanks!

Blake

*edit: oops, fixed gist links

[adinn]

Hi Blake,

Thanks for reporting this problem.

I copied your example into a repo on github and provided a pom in order to to test this.

When I run mvn test with jdk11 I get this output

$ jdk11
$ which java
~/redhat/openjdk/jdkupdates/jdk11u/images/jdk/bin/java
$ mvn test
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building bmsanity 1.0.0
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ bmsanity ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/adinn/tmp/byteman/bmsanity/src/main/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.1:compile (default-compile) @ bmsanity ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ bmsanity ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /home/adinn/tmp/byteman/bmsanity/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.1:testCompile (default-testCompile) @ bmsanity ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) @ bmsanity ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running BMSanityCheckTest
Setting org.jboss.byteman.allow.config.update=true
BMUnit : loading text script = BMSanityCheckTest+name
TransformListener() : handling connection on port 9091
org.jboss.byteman.agent.Transformer : possible trigger for rule rule in class BMSanityHelper
RuleTriggerMethodAdapter.injectTriggerPoint : inserting trigger into BMSanityHelper.doSomething() void for rule rule
org.jboss.byteman.agent.Transformer : inserted trigger for rule in class BMSanityHelper
Rule.execute called for rule_0
HelperManager.install for helper class org.jboss.byteman.rule.helper.Helper
calling activated() for helper class org.jboss.byteman.rule.helper.Helper
Default helper activated
calling installed(rule) for helper classorg.jboss.byteman.rule.helper.Helper
Installed rule using default helper : rule
rule execute
ack
something
BMUnit : unloading text script = BMSanityCheckTest+name
TransformListener() : handling connection on port 9091
retransforming BMSanityHelper
HelperManager.uninstall for helper class org.jboss.byteman.rule.helper.Helper
calling uninstalled(rule) for helper class org.jboss.byteman.rule.helper.Helper
Uninstalled rule using default helper : rule
calling deactivated() for helper classorg.jboss.byteman.rule.helper.Helper
Default helper deactivated
TransformListener() : handling connection on port 9091
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.537 s - in BMSanityCheckTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.011 s
[INFO] Finished at: 2019-04-10T17:07:03+01:00
[INFO] Final Memory: 9M/34M
[INFO] ------------------------------------------------------------------------

So, it works for me :-p

Actually, I'd very much like to get to the bottom of why it is not working for you. As you can see from the pom I'm using Byteman 4.0.6, JUnit 4.8.2. My runtime env employs Apache maven 3.3.3 (downloaded from the apache site) and a release of the latest jdk11u built from scratch on my machine.

It might be worth looking at the pom and checking the configuration to see if other config settings might be affecting the outcome. I'm also particularly interested to know where you got your jdk11 from?

And of course, you can clone my repo and see what happens when you use it. That might reveal whether the problem lies in the mvn/JUnit config or with your JDK/maven version.

It's definitely not a problem with jdk.attach.allowSelfAttach because the Byteman agent is clearly being loaded. If you omit it you get a message about not being able to load the agent.

Let me know whatever you find.

regards,

Andrew Dinn

[adinn]

PS apologies for the slow response -- I was delayed both in moderating and replying to your post because of a change to the login protocol at jboss.org

regards,

Andrew Dinn

[bdeggleston]

No problem! The login flow (which I guess is new) is pretty confusing, and had me going in circles for longer than I'd like to admit after creating my account yesterday.

Ok, so it turns out I'd had BYTEMAN_HOME set in my .bashrc for another test suite, and it was pointing to Byteman 3.0.10. So I'm assuming it was using that agent instead of the 4.0.6 one. That would also explain why the debugger was misbehaving. I'm not sure if it would make sense for BMUnit to log a warning when it runs into an agent version it's not expecting. It looks like the 4.x agent does print it's jar location though.

I'm using jdk-11.0.2 downloaded from the oracle site.

[adinn]

Hi Blake,

bdeggleston  wrote:
 Ok, so it turns out I'd had BYTEMAN_HOME set in my .bashrc for another
test suite, and it was pointing to Byteman 3.0.10. So I'm assuming it was
using that agent instead of the 4.0.6 one. That would also explain why the
debugger was misbehaving. I'm not sure if it would make sense for BMUnit
to log a warning when it runs into an agent version it's not expecting. It looks
like the 4.x agent does print it's jar location though.

Good, I'm glad to hear that is all it was.

One of the things you can do to avoid this problem is to unset BYTEMAN_HOME in your surefire plugin configuration.  You can find instructions on how to configure the plugin in the first BMUnit tutorial. Scroll down to the subsection entitled Recommended Surefire Settings.

I am not sure how BMUnit would  detect a problem like this. If it finds a setting for BYTEMAN_HOME it has to use the Byteman agent jar from that location. However, the BMUnit code doesn't currently have any way of identifying where it was loaded from (it might also be from BYTEMAN_HOME) nor does it know what its release version is or the agent jar's release version is.

Hmm? I guess in future releases I could inject a release version number into the agent and BMUnit code or, maybe, just write it into a properties file stored as a resource in each jar. If I get the agent to assign a System property with this release version String at agent init and get the BMUnit code to check the property after installing the agent then that would provide the necessary sanity check. That would work for new BMUnit releases mismatched with new agent releases. It would also work with new BMUnit releases mismatched with old agent releases that failed to set the property. Ooh, that's a nice idea! Thank you for the suggestion. I have raised JIRA issue BYTEMAN-381 to cover this bug. It should be fixed ready for the next release.

Do let me know if you have any further problems. Happy hacking!

regards,

Andrew Dinn