Cannot seem to trigger a rule against java.lang.ClassLoader#initLibraryPaths() method

[jaikiran]

Recently I was using Byteman to test one of the classloading issues. A couple of byteman rules (among many other) were:

RULE Classloader init library start
CLASS java.lang.ClassLoader
METHOD initLibraryPaths
AT ENTRY
IF TRUE
DO traceStack("*** init library paths called");
ENDRULE

RULE Classloader init library end
CLASS java.lang.ClassLoader
METHOD initLibraryPaths
AT EXIT
IF TRUE
DO traceStack("*** init library paths exited");
ENDRULE

These 2 rules try to check who calls the java.lang.ClassLoader#initLibraryPaths() method. This initLibraryPaths() is a package private static method on the ClassLoader class and is (only) called from the java.lang.System#initializeSystemClass() class. During booting of the JVM, I could see that this rule is identified by Byteman and applied without any errors (as per the logs. Sorry I lost hold of those logs, so can't share now). However, these 2 rules never get triggered, although I'm 100% certain that this initLibraryPaths() is indeed invoked (based on the fact that the rest of the code in this class uses certain state setup by this method). Do note that I have setup byteman startup property to "transform.all" (since this is a java.lang class) and am sure that's working since some other rules against this same java.lang.ClassLoader do work.

Is this maybe because this initLibraryPaths() method gets called too early before Byteman agent can intercept this call? I haven't yet had a chance to dig deeper and it might be a while before I get a chance to come back and debug this more, so just checking here. I'm on OpenJDK 8u242b08.

[adinn]

Hi Jaikiran

jaikiran  wrote:

. . .

Is this maybe because this initLibraryPaths() method gets called too early before Byteman agent can intercept this call? I haven't yet had a chance to dig deeper and it might be a while before I get a chance to come back and debug this more, so just checking here. I'm on OpenJDK 8u242b08.

I don't know for sure but I think this is extremely likely. If you switch on verbose trace and see logging saying the rule has been injected with no accompanying errors then I see no other reason why the injected would not get run. These are very simple rules which have been shown to work in many other injection contexts and so it is extremely unlikely that the injection would be failing silently.

Also, given how you describe its operation above I'd assume that the call from System has to happen before the Byteman agent jar can be loaded. This will not be th eonly such case. Not surprisingly, there are a whole host of things the JDK runtime needs to execute before an agent can safely be loaded,-- more before it can safely be started. All those actions are always going to be impossible to intercept via Byteman rules.

n.b. there is also a small set of methods that cannot be injected into even after loading Byteman. That's so because of the danger of entering a recursive triggering loop. Byteman does its best to detect recursive entry into injected code. If that happens from code which manages rule injection/execution it backs out of the injected trigger code. Clearly, the code which performs this check is unable to avoid such recursion. Byteman finesses this by blacklisting the methods used to perform the check. (essentially a ThreadLocal lookup).

regards,

Andrew Dinn