Accessing objects of private class

[bluegol]

Hi all,

I am trying to trace how servlet handles requests. My rule is:

RULE servlet start in FilterChain

INTERFACE javax.servlet.FilterChain

METHOD void doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse)

AT ENTRY

IF TRUE

DO

  startWithTarget(TT_SERVLET, $0, $1, $2);

ENDRULE

where startWithTarget is some function which handles tracing, TT_SERVLET some short constant value.

Now with Tomcat, the following exception occurs:

java.lang.IllegalAccessError: tried to access class org.apache.catalina.core.ApplicationFilterChain from class com.brainzsquare.apm.agent.RuleHelper_HelperAdapter_Compiled_81

  com.brainzsquare.apm.agent.RuleHelper_HelperAdapter_Compiled_81.execute0(test_rules.btm:13)

  com.brainzsquare.apm.agent.RuleHelper_HelperAdapter_Compiled_81.execute(test_rules.btm)

  org.jboss.byteman.rule.Rule.execute(Rule.java:777)

  org.jboss.byteman.rule.Rule.execute(Rule.java:746)

And googling org.apache.catalina.core.ApplicationFilterChain, reveals that $0 is an object of package private class. Seems fair.

Wait, this means that my rule body must be executed out of the method it targets. OK, I suspected that much.

NOW, I don't need all of the object $0. I just need to save it somewhere and compare it with other objects. In other words,

I need an "identity" of $0. But anyway, since byteman code is injected into the method, there must be a way to get my hands on $0.

So my next try is with the rule:

RULE servlet start in FilterChain

INTERFACE javax.servlet.FilterChain

METHOD void doFilter(javax.servlet.ServletRequest, javax.servlet.ServletResponse)

AT ENTRY

BIND o:Object = $0

IF TRUE

DO

  startWithTarget(TT_SERVLET, o, $1, $2);

ENDRULE

which unfortunately gives the same exception. Reflection doesn't help me either, because

calling a method with $0 results in the same exception.

Is there a way to get a grab of $0, e.g., as a plain java Object, or to extract some identity of $0 at least,

when $0 is an object of package private class?

Best,

Jahwan

P.S. Since the rule is on "INTERFACE javax.servlet.FilterChain", shouldn't $0 be just of type javax.servlet.FilterChain?

[adinn]

Hi Jahwan,

This is a known problem which has several manifestations (BYTEMAN-318, BYTEMAN-288, BYTEMAN-236 ). It has been partially fixed (BYTEMAN-319) but is still awaiting a proper fix.

As you have surmised rule code is executed out of line -- that's true both when a rule is interpreted and when it is compiled to bytecode. But that's not actually the cause of the problem because the same problem could arise even if the code were generated inline (imagine you try to access this.bar.baz where baz is package private and in a different package to this).

The problem is that the bytecode generated from a rule cannot refer to non-public types explicitly -- it must handle them generically (say as instances of Object) and operate on them via reflection. The current type checker and rule compiler do get this right some of the time but they are are not smart enough to get it right most of the time. In particular, the need to enforce type safety in the general case means that the rule compiler usually has to plant checkcast instructions when it obtains a value. Of course, that will fail to verify when the class for the checkcast is not visible to the class which owns the bytecode. Your attempt to get round this problem by using a BIND variable is a nice try but it founders because of the need for a checkcast when  the BIND variable is intiialised.

The workaround for this is to disable compilation for specific rules which end up employing non-public types by adding the clause

  NOCOMPILE

in the rule body (in the same location you would place a HELPER clause). The interpreter always treats objects generically and uses reflection so it has no access problems. That's not the best situation since it will slow down all cases where the rule is injected rather than just the ones which run up against non-public types. BYTEMAN does automatically disable compilation in some cases where it detects problematic use of non-public types (tta's the partial fix I mentioned above) but it does not yet detect all cases. The plan is first to extend the type checker to perform automatic disabling of compilation in all cases where the current compiler will fail and then to tackle the harder problem of implementing completely generic access to non-public data from generated bytecode (the work done for the first task is critical to being able to do the second).

It's also very astute of you to ask why $0 is typed to the trigger class ApplicationFilterChain rather than the target class (or rather interface in this case) FilterChain. This is a deliberate policy choice I made when implementing Byteman. Obviously, as far as $0 is concerned this only applies for interface and overriding rules where the trigger class (teh one being injected into) and target class (the one mentioned in the CLASS or INTERFACE clause) may differ. Choosing the trigger class as the type for $0 allows code in the rule body to do things with the target instance which would not be possible if you retained the generic target class as the type. For example a rule injected into INTERFACE ^List might include a to call $0.pop in the rule body. This is generally safe because type checking ensures that more specific operations are only attempted in cases where the trigger type actually implements them. So, with the List example type checking effectively constrains the rule only to apply to implementations which implement pop. You could just specify LinkedList but you might not want to rule out some related List implementation not derived from LinkedList (e.g. assume there was some other class called ArrayStack which implemented List and had a method called pop)

This may sound a bit of an odd option but it is not the only circumstance where code in the rule body may restrict the target to some subset of available trigger classes. For example, if you specify METHOD foo then a reference to $2 will means that foo(int) is not a valid trigger method (no 2nd parameter exists) and a reference to $2.name will rule out method foo(Foo, int) (parameter 2 is not an Object type with field name). Similarly, reference to local var $i will invalidate a potential trigger method with no such local variable.

[bluegol]

Hi Andrew,

Thanks for the explanation and workaround. Workaround with NOCOMPILE works for the moment.

It's also very astute of you to ask why $0 is typed to the trigger class ApplicationFilterChain rather than the target class (or rather interface in this case) FilterChain. This is a deliberate policy choice I made when implementing Byteman. Obviously, as far as $0 is concerned this only applies for interface and overriding rules where the trigger class (teh one being injected into) and target class (the one mentioned in the CLASS or INTERFACE clause) may differ. Choosing the trigger class as the type for $0 allows code in the rule body to do things with the target instance which would not be possible if you retained the generic target class as the type. For example a rule injected into INTERFACE ^List might include a to call $0.pop in the rule body. This is generally safe because type checking ensures that more specific operations are only attempted in cases where the trigger type actually implements them. So, with the List example type checking effectively constrains the rule only to apply to implementations which implement pop. You could just specify LinkedList but you might not want to rule out some related List implementation not derived from LinkedList (e.g. assume there was some other class called ArrayStack which implemented List and had a method called pop)

This may sound a bit of an odd option but it is not the only circumstance where code in the rule body may restrict the target to some subset of available trigger classes. For example, if you specify METHOD foo then a reference to $2 will means that foo(int) is not a valid trigger method (no 2nd parameter exists) and a reference to $2.name will rule out method foo(Foo, int) (parameter 2 is not an Object type with field name). Similarly, reference to local var $i will invalidate a potential trigger method with no such local variable.

IMHO, I'd rather leave $0 as the target class than convert it to the trigger class. The user should know what to do with $0, and she/he can always downcast or even use with reflection, if she/he so desires (and of course knows what she/he's doing.) There should be at least an option not to intrude into the code too much. I understand how the rule body restrict the target class, and so a user must be careful with what she/he is doing. It'd be great if a tool can sometimes restrict and  sometimes allow what to do exactly as prescribed by the user. Of course, more option means more complexity... Just my two cents.

[adinn]

Hi Jahwan,

Jahwan Kim wrote:

IMHO, I'd rather leave $0 as the target class than convert it to the trigger class. The user should know what to do with $0, and she/he can always downcast or even use with reflection, if she/he so desires (and of course knows what she/he's doing.) There should be at least an option not to intrude into the code too much. I understand how the rule body restrict the target class, and so a user must be careful with what she/he is doing. It'd be great if a tool can sometimes restrict and  sometimes allow what to do exactly as prescribed by the user. Of course, more option means more complexity... Just my two cents.

Yes, I think this decision was probably a mistake. If I were to recreate Byteman from scratch now I think I would probably stick to using the target class to type $0. However, I'm not really in a position to make that change since I cannot risk breaking existing tests that rely on the tyep being that of the underlying trigger class.

I do like your suggestion that I provide a mode where this can be disabled so $0 is strictly typed to the target type. I am not sure that I can modify the type checker to support this use but I will take a look at whether it is possible. I have raised a feature request JIRA BYTEMAN-325 to cover this option. If it can be implemented then it will certainly bypass many of the issues which arise because of private implementations of public super-types or interfaces.

Thank you very much for this very helpful suggestion. Thanks also for using Byteman and reporting your experiences with it. It's an enormous luxury to be able to receive user feedback, especially so when, as here, it generates new ideas for improving the project.

regards,

Andrew Dinn

[bluegol]

Hi Andrew,

Thanks for taking my suggestion. I may have some more requests, and I'd be more than glad to take part. After I finish my

current task of tracing all basic stuff, I will take a closer look at the byteman codes.

I'm also very glad that I made the right choice of taking byteman.

Regards,

Jahwan