What does the Byteman rulecheck maven plugin do?
Byteman release 2.1.3 introduced a maven plugin which runs the offline rule parser and type checker before executing any of your maven tests. This pluin allows you to be sure that the Byteman scripts used to test your application are still valid every time you modify, rebuild and retest your code.
Of course, you can always manually run from your favourite shell the bmcheck script which uses the ofline parser/type-checker to look for errors in your rules. bmcheck will also print warnings if it cannot find a target class to inject your rule into. The maven plugin improves on this by running the type checker automatically every time you build your application and failing your maven test run if the type checker notifies errors or unexpected warnings.
How would I use the plugin?
Let's answer that question by providing a simple example of a Byteman-based test where the plugin catches an error introduced during update of the application source. The example is a maven project comprising a single application class, a single test class and a Byteman script containing one rule. At the end of the tutorial a link is provided to a git project containing 3 versions of the project. The first shows the original working test. The second is the edited version which introduces an error. The third is the edited version but this time with the Byteman plugin configured so that the error gets caught during the test phase.
The original working example
Here is the rule used by the test
RULE check return result is true
CLASS MyClass
METHOD myMethod(int)
AFTER CALL MyClass.myOddMethod 2
IF NOT $!
DO THROW new RuntimeException("expected result true!")
ENDRULE
The location specifier for the rule says that it should be injected into MyClass.myMethod(int) after the second textual occurence of a call to MyClass.myOddMethod.
Now hereis the definition of the class the rule is being used to test
package org.my;
class MyClass
{
boolean flip = true;
boolean myMethod(int value)
{
flip = !flip;
boolean result;
if (flip) {
result = myOddMethod(2 * value);
} else {
result = myOddMethod(2 * value + 1);
}
return result;
}
boolean myOddMethod(int oddOrEven)
{
return ((oddOrEven & 1) == 1)
}
}
The test method myMethod contains two calls to myOddMethod so the rule gets injected into the else clause just after myOddMethod returns. Since the argument is always odd this call should always return true. The test checks the return value (referenced by the Byteman special variable $!) and throws an exception if it is not true. If you were to compile this class and run the bmcheck script you would see the following output
[adinn@zenade]$ bmcheck.sh -p org.my -cp . myScript.btm
Checking rule check return result is true against class MyClass
Parsed rule "check return result is true" for class MyClass
Type checked rule "check return result is true"
TestScript: no errors
[adinn@zenade]$
Editing the code introduces an error
Now let's assume that someone rewrites myMethod as follows in order to make it more concise
boolean myMethod(int value)
{
return myOtherMethod(2 * value + (flip ? 0 : 1));
}
Oh, dear! This has made the code shorter but it also deleted the line which inverted the boolean field flip. So, the test should now fail where before it succeeded.
However, the edit also deleted the second call to myOtherMethod. So, when the test is run the Byteman rule will no longer be injected because there is only one call to myOtherMethod. This failure to inject a rule raises a warning but not an error. That's because Byteman does not know if this is really an error. There may be some other class called MyClass defined in another package or another class loader where the rule does succcessfully match the code. The upshot is that the code is wrong but the test fails to find the error.
It is perfectly possible to detect errors like this. If you run bmcheck by hand you will see a warning like the following:
[adinn@zenade]$ bmcheck -cp . -p org.my myScript.btm
Checking rule check return result is true against class MyClass
WARNING : Problem type checking rule "check return result is true" loaded from myScript.btm line 5 against method myMethod(int) boolean
org.jboss.byteman.rule.exception.TypeWarningException: no matching injection point for method myMethod(int) boolean
WARNING : Problem type checking rule "check return result is true" loaded from myScript.btm line 5
org.jboss.byteman.rule.exception.TypeWarningException: failed to find any matching trigger method in class MyClass
TestScript: 2 total warnings
2 type warnings
[adinn@zenade]$
Hmm, not sure about that duplicate warning but at least you know there is something wrong!
Of course, if you forget to run bmcheck and build your project with maven the warning will merely get ignored and your test will pass the code even though it is wrong. Wouldn't it be nice if maven could run the bmcheck for you and tell you if there were errors or warnings from your script . . .
Byteman rulecheck maven plugin to the rescue!
So that' s exactly what the new Byteman rulecheck maven plugin does for you. Here's how you would configure your pom to include the plugin and tell it how to find your test classes and scripts:
<project>
. . .
<build>
. . .
<plugins>
<plugin>
<groupId>org.jboss.byteman</groupId>
<artifactId>byteman-rulecheck-maven-plugin</artifactId>
<version>2.1.3</version>
<executions>
<execution>
<id>rulecheck-test</id>
<goals>
<goal>rulecheck</goal>
</goals>
<configuration>
<packages>
<package>org.my</package>
</packages>
<includes>
<include>**/*.btm</include>
</includes>
<verbose>true</verbose>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
. . .
The packages entry in the configuration says to use the prefix org.my when trying to resolve the name MyClass found in the rule's CLASS clause. The includes entry says to check any files found below the project test tree which contain a .btm suffix. Finally, the verbose flag configures printing of various progress messages from the plugin so you can see it working when mvn runs it.
So, now, with the plugin configured, the type checker is run by maven as a subgoal of the test phase and the presence of warnings causes it to fail.
There are several other configuration options which can be provided:
- fallOnError (default true) can be set to false if you want tests still to run even if there are parse or type errors in your rules
- failOnWarning (default true) can be set to false to allow tests still to run when the check produces only warnings
- expectedWarnings (default 0) can be set to an expected warning count and tests will only proceed if exactly that number of warnings is generated
- skip (default false) can be set to true if you want the plugin not to run
Can I try this at home?
You can download three example projects which show the code in this article being tested. Directory example1 contains the original working version. Directory example2 contains the broken version which still manages to build because the Vyteman rule fails to be injected.Finally, example3 includes the version where the plugin is configured and it traps the error, failing the tests. You can exercise the test by running command mvn test in each directory. Here is the output when you run the broken version in directory example3:
[adinn@zenade example3]$ mvn test
[INFO] Scanning for projects...
. . .
[INFO] Compiling 1 source file to /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes
[INFO]
[INFO] --- byteman-rulecheck-maven-plugin:2.1.3-SNAPSHOT:rulecheck (rulecheck-test) @ plugin-example ---
[INFO] find byteman script in /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes
[INFO] /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/classes
[INFO] /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.19.x86_64/jre/../lib/tools.jar
[INFO] /ssd/home/adinn/.m2/repository/junit/junit/4.8.2/junit-4.8.2.jar
[INFO] /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/classes
[INFO] /ssd/home/adinn/.m2/repository/junit/junit/4.8.2/junit-4.8.2.jar
[INFO] /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/classes
[INFO] /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes
[INFO] add script /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes/myScript.btm
[INFO] add package org.my
[INFO] Checking 1 byteman scripts in /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes
[WARNING] WARNING : Problem type checking rule "check return result is true" loaded from /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes/myScript.btm line 5 against method myMethod(int) boolean
org.jboss.byteman.rule.exception.TypeWarningException: no matching injection point for method myMethod(int) boolean
[WARNING] WARNING : Problem type checking rule "check return result is true" loaded from /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes/myScript.btm line 5
org.jboss.byteman.rule.exception.TypeWarningException: failed to find any matching trigger method in class org.my.MyClass
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.893s
[INFO] Finished at: Mon Jun 24 18:20:40 BST 2013
[INFO] Final Memory: 15M/236M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.jboss.byteman:byteman-rulecheck-maven-plugin:2.1.3-SNAPSHOT:rulecheck (rulecheck-test) on project plugin-example: check byteman script rules failed with 2 warnings. You may config failOnWarning with false or expectWarnings with 2 -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[adinn@zenade example3]$
If you go back and correct the rule by changing AT CALL myOtherMethod 2 to AT CALL myOtherMethod it will inject the rule at the first (and only) call to myOtherMethod and the test will run and catch the error:
[adinn@zenade example3]$ mvn test
[INFO] Scanning for projects...
. . .
[INFO] --- byteman-rulecheck-maven-plugin:2.1.3-SNAPSHOT:rulecheck (rulecheck-test) @ plugin-example ---
[INFO] find byteman script in /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes
[INFO] /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/classes
[INFO] /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.19.x86_64/jre/../lib/tools.jar
[INFO] /ssd/home/adinn/.m2/repository/junit/junit/4.8.2/junit-4.8.2.jar
[INFO] /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/classes
[INFO] /ssd/home/adinn/.m2/repository/junit/junit/4.8.2/junit-4.8.2.jar
[INFO] /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/classes
[INFO] /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes
[INFO] add script /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes/myScript.btm
[INFO] add package org.my
[INFO] Checking 1 byteman scripts in /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.7.1:test (default-test) @ plugin-example ---
[INFO] Surefire report directory: /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.my.MyTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.407 sec <<< FAILURE!
Results :
Tests in error:
myTest(org.my.MyTest)
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.378s
[INFO] Finished at: Mon Jun 24 18:33:41 BST 2013
[INFO] Final Memory: 10M/297M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.7.1:test (default-test) on project plugin-example: There are test failures.
[ERROR]
[ERROR] Please refer to /ssd/home/adinn/jboss/byteman/git/byteman-rulecheck-maven-plugin-example/example3/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[adinn@zenade]$
Comments