BMUnit : Using Byteman with JUnit or TestNG from maven and ant

Version 12

Created by adinn on Nov 10, 2014 5:23 AM. Last modified by adinn on Mar 21, 2019 8:27 AM.

Let's assume you know what Byteman is . . .

This is the Part Two (of three) in the Byteman tutorial series, explaining how you can use Byteman to improve the simplicity, quality and coverage of your unit, integration and system tests. This tutorial introduces BMUnit, the package which integrates Byteman with the two most popular Java test frameworks, JUnit and TestNG. It also shows you how to configure maven and ant to drive your BMUnit tests. Part One is a basic How-To which explains what Byteman is and tells you how to use Byteman from the command line to inject side-effects into Java programs. Part Three is a follow-up to this tutorial which uses a worked example application to show the full power of Byteman and BMUnit.

If you are new to Byteman then you probably ought to read part one before you continue with this article. If you already know what Byteman does then this article will help you take the next step, employing BMUnit to extend the range of simple JUnit tests to provide better tracing of test activity and perform validation and fault injection. This will prepare you for part three which will teach you how to cover test scenarios which would be very difficult or, in some cases, impossible to implement without the use of a tool like Byteman.

As with the other articles the material here is structured as a FAQ comprising answers to a series of questions. If you have not used Byteman with JUnit or TestNG or are not familiar with how it integrates into maven or ant then you should probably read through from start to finish trying out the examples. If you have written tests with Byteman already and just want to refresh your memory or answer a specific question, you can jump ahead using one of these links. However, in the latter case you might find it helpful to skim the preceding section(s) in order to fully grasp what is being presented.

What is BMUnit?
Is There A Simple Example Of How To Use BMUnit?
How Do I Inject Trace Into JUnit Tests?
How Can I Check My Rules Are Being Run
How Do I Configure Injection Into JUnit With Maven?
How Do I Configure Injection Into JUnit With Ant?
How Do I Inject Faults Into JUnit Tests?
How Do I Configure Injection Into TestNG With Maven?
How Do I Configure Injection Into TestNG With Ant?
Where Can I Download The Tutorial Sources?

What is BMUnit?

BMUnit is a package which makes it simple to use Byteman as a testing tool by integrating it into the two most popular Java test frameworks, JUnit and TestNG. Like both these test frameworks BMUnit is based around the use of annotations. By adding just a few annotations to your code and a few jars to the classpath you can ensure that the Byteman agent is automatically loaded into the JVM which runs your JUnit/TestNG tests. These annotations also specify Byteman rules to be installed by the agent during the test run, changing the way the application code or, indeed, the Java runtime operates. Rules can be loaded before a test is run and then unloaded after the test completes.

The most basic test rules inject trace code so you can see what is actually being executed. This allows you to be sure that your test is exercising the intended path through the code. More complex rules perform validation by specifying a condition which tests for an error situation. The condition ought to be false when the rule is triggered. If it turns out to be true then the rule action can throw an exception, causing the test to fail.

The most sophisticated rules employ a technique called fault injection. They modify the behaviour of the application or the runtime to simulate test scenarios which would only happen in unusual circumstances. For example, they might force the runtime to throw a FileNotFoundException when a specific file is opened or change the return value from a call to simulate an error. Basic tests may only need to employ trace rules or simple validation rules. More sophisticated tests will usually employ all three types of rule and may inject multiple rules in order both to create a fault and then propagate it through the application code. This tutorial provides simple examples of all three types of rule. Advanced fault injection and propagation are presented in the next tutorial.

BMUnit allows you to specify simple rules directly in annotations on test classes or test methods. Class annotations are used for rules which apply to all test methods in the class. These rules get loaded once before running all the methods and then unloaded after all the methods have been tested. This is particularly suitable for trace rules. You often need to trace the same information during test execution whatever test you are running. Sometimes it is also appropriate to specify validation rules in class annotations. Certain situations should never arise in any test.

Annotations on individual test methods cause rules to be loaded just for the extent of that specific test and unloaded when the test has competed. Method annotations are usually most appropriate when you are specifying a fault injection rule. It might be appropriate to throw an exception in some of your tests, say when they try to open a file. However, you will probably want the behaviour of the file stream classes to be restored to normal if your other tests actually need to test read and write operations.

Is There a Simple Example of How To Use BMUnit?

Yes, the example tests shown below are simple and easy to follow. But before showing them to you we need to introduce the sample application used throughout this tutorial so you can see what code is being broken and what is supposed to happen when you break it.

The application is a simple class called WebWriter. It creates a very basic HTML page which includes your name in the title and in the body text. You can also specify the file the HTML is to be written to. WebWriter includes a main method, methods used to manage opening and closing the output file stream and methods which construct and then write the various elements of the HTML content.

At the end of this article you will find details of how to download and install the application source and test code and the Byteman rule scripts plus the libraries required to test it with Byteman. The source is organised as a maven project comprising a top-level module, an application source sub-module and three test sub-modules. However, the source tree also contains ant build scripts allowing you to compile and package the sources and run the tests using ant. The rest of this tutorial will guide you through the steps required to perform each of these operations.

N.B. the example application shows you how BMUnit integrates Byteman with JUnit4. The application works with all Byteman 2.x.x, 3.x.x and 4.x.x releases and on all JDKs from JDK6 onwards. BMUnit has also been extended to integrate with JUnit5. However, for that to work you need to use Byteman release 4.0.6 or later and you can only run with a JDK8 or later release of Java (JUnit5 depends on classes that only exist in JDK8+). Instructions for configuring JUnit5 are provided below alongside the JUnit4 configuration.

Once you have downloaded and unzipped the application source take a look at subdirectory app. This contains the application library classes and some main program classes which exercise the library. If you want to build the library you can run command

> mvn package

. . .

in the top level directory. Alternatively, if you are using ant you can run command

> ant build

. . .

This should create a directory called target in subdirectory app in which you should find a jar called bmunit-tutorial-app-1.0.0.jar

To run the example applications change to the app directory and run the WebWriter main method providing a file name and a user name as arguments. For example,

> cd app

> java -classpath target/bmunit-tutorial-app-1.0.0.jar \

org.my.app.WebWriter foo.html Andrew

You should find file foo.html in the top level directory. Open it in a web browser and check that the title and body text contain the name Andrew.You can also try running the program with a different file name and user name to see that it changes.

How Do I Inject Trace Into JUnit Tests?

The sample application contains a subdirectory called junit, which includes a single test class WebWriterTest. It contains a single test method testWriteHead which creates a WebWriter and then calls method writeHeader.

@Test

public void testWriteHead()

{

System.out.println("-------- testWriteHead ---------");

WebWriter writer = new WebWriter("foo.html", "Andrew");

writer.writeHeader(System.out);

System.out.println("-------- testWriteHead ---------\n");

}

Notice that the PrintStream passed in the call to is writeHeader is System.out. So, it is possible to verify that this test has passed by eyeballing the printed output. However, it would be better if this check could be automated. That is where Byteman comes into play.

The Check Rule

The Byteman rule used in this test is contained in script check.btm in directory junit/src/resources

RULE check head

CLASS WebWriter

METHOD makeHeader

AT EXIT

BIND result = $builder.toString()

IF (NOT result.contains("<HEAD>") || NOT result.contains("</HEAD>"))

DO THROW new RuntimeException("invalid header format");

ENDRULE

This rule is injected into method WebWriter.makeHeader at the point where that method is about to return. That means the method will already have appended the header text into the supplied StringBuilder. The rule binds a local rule variable result, which it initializes by calling the StringBuilder's toString method. Notice that it uses the syntax $builder to refer to parameter variable builder which is still in scope at the return point. The rule condition is just a basic check to see that the header text looks like proper HTML. If the check fails the rule action throws an exception causing the test to fail.

Using BMUnitRunner to Inject the Rules

The rules in script check.btm are loaded automatically when the test is run by attaching some annotations to the test class:

@RunWith(org.jboss.byteman.contrib.bmunit.BMUnitRunner.class)

@BMUnitConfig(loadDirectory="target/test-classes")

@BMScript(value="check.btm")

public class WebWriterTest

{

. . .

The @RunWith annotation belongs ot the JUnit package and it is needed to tell JUnit to use the Byteman class BMUnitRunner to run the tests. BMUnitRunner is a subclass of the standard JUnit4 test runner class and it delegates most of the job of running the test to JUnit. However, it operates as a wrapper class, installing the Byteman agent into the test JVM when testing begins and loading and unloading Byteman rules as they are needed to run specific tests.

In this case the runner detects the presence of an @BMScript annotation on the test class. So, it requests the Byteman agent to install the rules from the script specified in the annotation check.btm. Note that maven copies any files found in the test/resources directory to directory target/test-classes before it starts running any tests so this script can be located using path "target/test-classes".

The @BMUnitConfig annotation is also detected by class BMUnitRunner. This annotation can be used to configure the operation of the BMUnit package. It can also be omitted, in which case sensible defaults will be employed. In this case the loadDirectory property is used to tell BMUnit where to look to load file scripts. So, this setting means that BMUnit will try to load the Byteman rule file target/test-classes/check.btm.

Once all tests in the class have been executed BMUnitRunner connects to the agent listener and unloads all the rules defined in the script. If instead the @BMScript annotation had been attached to the test method then the rules would be uploaded and unloaded just before and after running that specific test.

If you are using maven then you can run the tests in module junit from the root directory of your download tree by enabling the junit profile. The profile entry for profile junit in the top-level pom adds subdirectory junit as a module, ensuring that its tests get run as part of the top-level build process

> mvn -P junit test

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running org.my.WebWriterTest

byteman jar is /ssd/home/adinn/.m2/repository/org/jboss/byteman/byteman/2.2.1/byteman-2.2.1.jar

Setting org.jboss.byteman.allow.config.update=true

-------- testWriteHead ---------

<HEAD>

<TITLE>

Welcome to the web page for Andrew

</TITLE>

</HEAD>

-------- testWriteHead ---------

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.385 sec

. . .

If you are using ant then you can run the test from the top level directory with the following command

> ant junit

. . .

test:

[copy] Copying 1 file to /ssd/home/adinn/jboss/byteman/git/bmunit-tutorial/junit/target/test-classes

[junit] Setting org.jboss.byteman.allow.config.update=true

[junit] -------- testWriteHead ---------

[junit] <HEAD>

[junit] <TITLE>

[junit] Welcome to the web page for Andrew

[junit] </TITLE>

[junit] </HEAD>

[junit]

[junit] -------- testWriteHead ---------

[junit]

BUILD SUCCESSFUL

. . .

Note that for this ant command to work you must ensure that BYTEMAN_HOME is set and identifies the directory where you downloaded and unzipped the Byteman binary release. Also, make sure you have run command ant build first to ensure that the application classes have been compiled and installed in the application jar. Note also that the ant task which runs the test includes a copy task which copies the rule script into directory target/test-classes.

Configuring your test to run with JUnit5

If you want to run your test using JUnit5 then you simply use a different annotation:

@WithByteman

@BMUnitConfig(loadDirectory="target/test-classes")

@BMScript(value="check.btm")

public class WebWriterTest

{

. . .

The @WithByteman annotation belongs to package org.jboss.byteman.bmunit (the same one as all the other BMUnit API classes). However, in order to make it available at runtime you will need to add byteman-bmunit5.jar as an extra dependency in your classpath (see below for details of the maven or ant configuration).

How Can I Check My Rules Are Being Run

Of course the above test passes (because method makeHeader is correct) but there is no evidence that the rule is actually being injected and validating the output. You could check that it is workign correctly by breaking the test class, for example modifying makeHeader so that it does not insert the "<HEAD>" tag. However, that might be true for now but might change if you edit the code (for example changing the name of the target method). A more useful way to check that Byteman is running is to use its debug capability. The rule supplied in the example project is in reality slightly different to the text quoted above. Here is the full version:

RULE check head

CLASS WebWriter

METHOD makeHeader

AT EXIT

BIND result = $builder.toString()

IF debug("checking for HEAD and /HEAD tags") &&

(NOT result.contains("<HEAD>") || NOT result.contains("</HEAD>"))

DO debug("makeHeader result does not include both <HEAD> and </HEAD>");

THROW new RuntimeException("invalid header format");

ENDRULE

As you can see, the condition and action both include calls to method debug. This method does not normally output any text. If you enable debug output then you will see trace output from these calls to debug. n.b. method debug always returns true which means you can add debug trace calls to a condition by combining it with the rest of the condition using the && operator.

If you are using maven then you can enable debug by tweaking the settings in the @BMUnitConfig annotation:

@RunWith(org.jboss.byteman.contrib.bmunit.BMUnitRunner.class)

@BMUnitConfig(loadDirectory="target/test-classes", debug=true)

@BMScript(value="check.btm")

public class WebWriterTest

{

. . .

You can see from the output that the debug method in the condition has been executed. However, the debug message in the action is not executed because the condition turns out to be false, as expected. The other debug messages show rules being installed and uninstalled and show notifications to the associated helper classes to indicate that a rule using that helper class is active or has been deactivated.

n.b. if your test class does not include an @BMUnitConfig annotation you can still enable debug by configuring argument -Dorg.jboss.byteman.debug on the Java command line for the test JVM.

If you want to see even more information about what BMUnit and Byteman are doing behind the scenes there are two more detailed trace settings for you to enable. The @BMUnitConfig attributes bmunitVerbose=true and verbose=true enable detailed tracing of the operation of the BMUnit code and the Byteman agent code. The first setting enables trace of BMUnit installing the agent and uploading/unloading rules via the agent listener socket. The second setting enables trace of the agent itself, showing rules being injected into target methods and then executed when control reaches the trigger point.

How Do I Configure Injection Into JUnit With Maven?

So, how is the maven pom configured to ensure that the Byteman code is available? Well, you just need to include a few byteman jars as test dependencies and set a small number of the surefire plugin configuration properties. The file pom.xml in directory junit includes the following dependencies:

<artifactId>tutorial2-app</artifactId>

</dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

</dependency>

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman</artifactId>

<version>${byteman.version}</version>

</dependency>

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman-submit</artifactId>

<version>${byteman.version}</version>

</dependency>

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman-install</artifactId>

<version>${byteman.version}</version>

</dependency>

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman-bmunit</artifactId>

<version>${byteman.version}</version>

</dependency>

<artifactId>tools</artifactId>

<scope>system</scope>

<systemPath>${tools.jar}</systemPath>

</dependency>

</dependencies>

The byteman jars required in the path are byteman.jar, byteman-submit.jar, byteman-install.jar and byteman-bmunit.jar. You will need this version or later for the test to work correctly. The test also depends on the app jar and a junit 4 jar. Finally, it includes a dependency on the JVM tools jar which is necessary when running on the Oracle or OpenJDK JVMs or on JRockit. The tools jar provides classes needed in order to dynamically install the Byteman agent.

Note that property byteman.version is defined in the top-level pom as 2.2.1. and property tools.jar is defined in the top-level pom as ../lib/tools.jar.

Note also that to run on JDK9 you must omit the tools jar dependency. JDK9 has deleted the tools jar and moved the relevant classes into the base runtime image. If you look at the pom supplied with the project you can see how to configure your pom so it can run on both JDK9 and earlier JDKs.

Configuring maven to use JUnit5

If you want to use JUnit5 to run your tests then you need to add an extra dependency on byteman-bmunit5.

. . .

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman-bmunit5</artifactId>

<version>${byteman.version}</version>

</dependency>

. . .

In order for JUnit5 support to work correctly you will need to use Byteman version 4.0.6 or later and use JDK8 or later.

Recommended Surefire Settings

The surefire plugin usually runs Byteman tests ok with the default configuration but you should probably specify a few properties to ensure correct operation. Here is the configuration from pom in directory junit

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-surefire-plugin</artifactId>

<include>org/my/WebWriterTest.java</include>

</includes>

<parallel>false</parallel>

<BYTEMAN_HOME></BYTEMAN_HOME>

</environmentVariables>

<name>org.jboss.byteman.home</name>

</property>

</systemProperties>

<argline>-Djdk.attach.allowAttachSelf</argline>

</configuration>

</plugin>

The values for useSystemClassloader and useManifestOnlyJar are both the default settings but they are both necessary in order for the agent to operate properly.

The test must be run in a forked JVM so that the Byteman agent can be loaded correctly. Hence forkMode should be set either to once or always.

Tests should be run serially in order to avoid problems where rules for one test introduce side-effects which break code executed by an unrelated test. So, parallel should normally be specified with value false.

When loading the agent the BMUnit package checks environment variable BYTEMAN_HOME and System property org.jboss.byteman.home to see if either of them is set to a non-empty string. If so uses this as a path to locate a local download of byteman.jar. Otherwise it scans the classpath to locate the jar. If your maven pom is configured correctrly then the jar you need will be in the classpath. So, as a precaution these two variables can be cleared in the surefire configuration to ensure that the tests do not accidently pick up the wrong version of byteman.jar. If you are sure these variables are not set you can omit these entries from the configuration.

When running on JDK9 and later releases it is necessary to set System property jdk.attach.allowAttachSelf to true so that BMUnit can load the Byteman agent into the test JVM. Since this is a read-only property it must be set on the command line. If it is configured in the systemProperties configuration block maven will try to reset it before running the tests but the call to System.setProperty() will fail and BMUnit will be unable to install the agent into the JVM. If you are running on an earlier JDK release (6, 7 or 8) then you can omit this property setting from the configuration.

How Do I Configure Injection Into JUnit With Ant?

The ant build script build.xml in directory junit shows you how to configure ant to use Byteman. When running with ant you have to ensure the byteman jars are available by downloading and unzipping a Byteman release and setting BYTEMAN_HOME to identify the download directory. The script ensures that BYTEMAN_HOME has been set and that it identifies a directory containing the expected jar files

. . .

<not>

</not>

</condition>

</fail>

<not>

<and>

</and>

</not>

</condition>

</fail>

<not>

</not>

</condition>

</fail>

. . .

The BMUnit jar must be on the classpath when compiling the test code so that references to the byteman annotations and the Byteman test runner class can be resolved.

. . .

</classpath>

</javac>

Three of the four Byteman jars, byteman-bmunit.jar. byteman-submit.jar and byteman-install.jar, must be configured on the classpath when you run the JUnit tests. The agent jar, byteman.jar, does not need to be included in the classpath as BMUnit locates and installs it using environment variable BYTEMAN_HOME.

The junit task which runs the tests is configured as follows

</classpath>

</junit>

Notice that the required junit jar is provided as part of the zip download in the top level lib directory. Also note that the junit tests are forked into a separate JVM. This is needed to allow the Byteman agent to be loaded correctly.

Configuring ant to use JUnit5

If you want to run your tests from ant using JUnit5 then you need to ensure that jar byteman-bmunit5.jar is also in the classpath. You can do this by modifying the above configuration as follows:

        . . .
        <fileset dir="${byteman.home}/contrib/bmunit" includes="byteman-bmunit5.jar, byteman-bmunit.jar"/>
        <fileset dir="${byteman.home}/lib" includes="byteman-submit.jar byteman-install.jar/>

. . .

How Do I Inject Faults Into JUnit Tests?

Injecting faults is done in exacttly the same way as you inject trace or validation rules i.e. using annotations provided by the BMUnit package. However, in most cases you want fault rules to be injected before a specific test and then removed after the test has completed. This is because you normally only want to break the application or Java runtime in a way that is highly specific to the test in question. The junit2 subdirectory contains two further JUnit tests in class WebWriterTest2 which employ rules to break the application. Here is the first method:

@RunWith(org.jboss.byteman.contrib.bmunit.BMUnitRunner.class)

@BMUnitConfig(loadDirectory="target/test-classes")

@BMScript(value="check.btm")

public class WebWriterTest2

{

@Test

@BMRule(name = "handle file not found",

targetClass = "FileOutputStream",

targetMethod = "<init>(File)",

action = "throw new FileNotFoundException( \"Ha ha Byteman fooled you again!\" )"

)

public void handleFileNotFound()

{

System.out.println("-------- handleFileNotFound ---------");

WebWriter writer = new WebWriter("foo.html", "Andrew");

PrintStream ps = writer.openFile();

Assert.assertTrue(ps == null);

System.out.println("-------- handleFileNotFound ---------\n");

}

. . .

This @BMScript annotation on the class is still present in order to load the validation rule in check.btm. However, the test method has a new annotation, @BMRule, which specifies a rule to be uploaded just before the test is run and unloaded immediately after it has completed. This annotation provides the details of the rule directly in the annotation attributes rather than placing them in a separate script. The target class for the rule is class FileOutputStream and the target method is the constructor which creates an outputstream from a given File instance (n.b. Byteman follows the JVM's internal naming convention which uses <init> as the name for constructor methods and <clinit> as the name for class static initialiser methods). The annotation does not specify a location or a condition so they default to "AT ENTRY" and "TRUE", respectively. The action is specified and it creates and then throws a FileNotFoundException. So, when the test is run this rule will throw an exception when method WebWriter.openFile tries to open an output stream. The test will only succeed if the call to openFile handles this exception and returns a null result.

The second test also uses an @BMRule annotation to side effect class FileOutputStream

. . .

@Test

@BMRule(name = "test write body",

targetClass = "FileOutputStream",

targetMethod = "write(byte[],int,int)",

condition = "incrementCounter($0) == 2",

action = "throw new java.io.IOException( \"Ha ha Byteman fooled you!\" )"

)

public void testWriteBody()

{

System.out.println("-------- testWriteBody ---------");

WebWriter writer = new WebWriter("foo.html", "Andrew");

PrintStream ps = writer.openFile();

boolean result = writer.writeHeader(ps);

Assert.assertTrue(result);

result = writer.writeBody(ps);

ps.close();

Assert.assertFalse(result);

System.out.println("-------- testWriteBody ---------\n");

}

In this case the target is the write method used to flush a block of text to the output file so the rule will be triggered whenever a block of text is inserted by the application. The action throws an IOException so the rule is intended to test the ability of the application to handle a write error. The condition uses a counter to ensure that the rule is only fired at the second triggering. The counter is identified using $0, a parameter variable which references the FileOutputStream instance for the trigger method call (it could also refer to it as $this).

At the first call to method write when the rule is triggered the counter is automatically created and given the default value 0. The call to incrementCounter in the condition increments this counter and returns 1. The condition is false so the rule action is not executed and execution of method write continues. At the second call the same counter is identified and this time incrementCounter returns 2 making the condition true.

When the test is run the call to writeHeader should successfully flush the first block of header text to the file. The call to writeBody should fail to write the body text, catch the exception and return a status indicating that it has failed. The code in method testWriteBody checks for both of these outcomes.

Using BMUnitRunner to Inject the Fault Rules

The tests in directory junit2 can be run by executing the following command

> mvn test -P junit2

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running org.my.WebWriterTest2

byteman jar is /ssd/home/adinn/.m2/repository/org/jboss/byteman/byteman/2.2.1/byteman-2.2.1.jar

Setting org.jboss.byteman.allow.config.update=true

-------- testWriteBody ---------

Error writing body

-------- testWriteBody ---------

-------- handleFileNotFound ---------

Unable to open file foo.html

java.io.FileNotFoundException: Ha ha Byteman fooled you again!

-------- handleFileNotFound ---------

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.398 sec

. . .

If you are using ant you can execute the JUnit fault injection tests from the top level directory wiht the following command

> ant junit2

. . .

You should see essentially the same output as when you run the maven version of the test.

How Do I Inject Rules Into TestNG Tests?

The BMUnit package can be used with TestNG in almost exactly the same way as it is used with JUnit. The only major difference is in how you inform TestNG that the test requires the use of Byteman rules. We saw that with JUnit you need to add an @RunWith annotation to your test class. For TestNG based tests there are two ways to achieve this. One way is to make your test class inherit from a BMUnit class called BMNGRunner. This parent class provides all the behaviour needed to ensure that rules are loaded before running tests and unloaded after tests complete. If you cannot reparent your test class then you can use the TestNG @Listeners annotation to tell TestNG that your test needs to be managed by BMUnit.

The testng sub-directory contains a test class WebWriter3 which is essentially the same as WebWriter2:

@BMUnitConfig(loadDirectory="target/test-classes")

@BMScript(value="check.btm")

public class WebWriterTest3 extends BMNGRunner

{

@Test

@BMRule(name = "handle file not found",

targetClass = "FileOutputStream",

targetMethod = "<init>(File)",

action = "throw new FileNotFoundException( \"Ha ha Byteman fooled you again!\" )"

)

public void handleFileNotFound()

{

. . .

TestNG recognises the superclass BMNGRunner and invokes its wrapper methods to install the byteman agent and upload/unload Byteman rules specified in the @BMScript and @BMRule annotations. Note that in the above example the @Test annotation on method handleFileNotFound refers to the annotation in package org.testng.annotations rather than the JUnit annotation in package org.junit.

The same result could be achieved without subclassing your test class using the @Listeners annotation to attach an instance of class BMNGListener. The listener instance delegates control to class BMNGRunner.

@BMUnitConfig(loadDirectory="target/test-classes")

@BMScript(value="check.btm")

@Listeners(BMNGListener.class)

public class WebWriterTest

{

. . .

Using BMNGRunner to Inject the Fault Rules

To run the tests using maven you execute the following command in the top level directory

> mvn test -P testng

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running TestSuite

byteman jar is /ssd/home/adinn/.m2/repository/org/jboss/byteman/byteman/2.2.1/byteman-2.2.1.jar

Setting org.jboss.byteman.allow.config.update=true

-------- handleFileNotFound ---------

Unable to open file foo.html

java.io.FileNotFoundException: Ha ha Byteman fooled you again!

-------- handleFileNotFound ---------

-------- testWriteBody ---------

Error writing body

-------- testWriteBody ---------

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.064 sec

. . .

Alternatively you can execute the following ant command

> ant testng

. . .

How Do I Configure Injection Into TestNG With Maven?

The TestNG configuration is very much the same as for JUnit except that a dependency on TestNG is required instead of a dependency on JUnit.

<artifactId>tutorial2-app</artifactId>

</dependency>

<groupId>junit</groupId>

<artifactId>junit</artifactId>

</dependency>

<groupId>org.testng</groupId>

<artifactId>testng</artifactId>

<scope>compile</scope>

</dependency>

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman</artifactId>

<version>${byteman.version}</version>

</dependency>

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman-submit</artifactId>

<version>${byteman.version}</version>

</dependency>

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman-install</artifactId>

<version>${byteman.version}</version>

</dependency>

<groupId>org.jboss.byteman</groupId>

<artifactId>byteman-bmunit</artifactId>

<version>${byteman.version}</version>

<groupId>org.testng</groupId>

<artifactId>testng</artifactId>

</exclusion>

</exclusions>

</dependency>

<artifactId>tools</artifactId>

<scope>system</scope>

<systemPath>${tools.jar}</systemPath>

</dependency>

Note that the same caveats apply as for the JUnit case. Property byteman.version is defined in the top-level pom as 2.2.1. and property tools.jar is defined in the top-level pom as ../lib/tools.jar.

Note also once again that to run on JDK9 you must omit the tools jar dependency. JDK9 has deleted the tools jar and moved the relevant classes into the base runtime image. If you look at the pom supplied with the project you can see how to configure your pom so it can run on both JDK9 and earlier JDKs.

Note that the TestNG version specied here is the earliest which has been found to work.

The surefire configuration is essentially the same as for the JUnit tests

<groupId>org.apache.maven.plugins</groupId>

<artifactId>maven-surefire-plugin</artifactId>

<include>org/my/WebWriter3.java</include>

</includes>

<parallel>false</parallel>

<BYTEMAN_HOME></BYTEMAN_HOME>

</environmentVariables>

<name>org.jboss.byteman.home</name>

</property>

</systemProperties>

</configuration>

</plugin>

How Do I Configure Injection Into TestNG With Ant?

TestNG provides an ant task allowing you to run TestNG tests which requires much the same configuration options as for runnign a JUnit test. So, the compile and test targets in the TestNG ant build script look much the same as the ones in the JUnit build scripts

. . .

. . .

</classpath>

</javac>

</target>

</classpath>

</testng>

</target>

The taskdef declaration is necessary to make the testng task available.

Notice that the required junit and testng jars are provided as part of the zip download in the top level lib directory. It is not possible to run the TestNG tests from ant using version 5.14.6, the one used in the maven tests. However, it is possible to run tests from ant using version 6.3.1.

Where Can I Download The Tutorial Sources?

The tutorial sources are available from the JBoss downloads server as a zip file. You can also access them from github.

In order to run the example application main programs and tests provided as part of this downlaod you will need to install a JDK6 (or greater) release which supports the dynamic agent load capability. This includes OpenJDK, Oracle's JDK and recent releases of the JRockit JDK (version jrmc-4.0.1- is known to work). You should ensure that the Java binaries in the JDK bin directory are available in your PATH.

You will also need to install either ant or maven, This tutorial has been tested using ant 7.1 and maven 3.0.2 but it may possibly work with earlier versions. Once again you should ensure the ant or maven binaries are available in your path.

If you are building and running the test programs with maven you do not need to downlaod and install a Byteman release as maven will automatically pull the relevant jars from the network. Byteman jars are available in the Maven Central repository so you do not need any special configuration in order to be able to download them.

If you are building and running the test programs with ant then you will need to download Byteman release 2.2.1 or greater. The latest release is available from the Byteman downloads page

http://www.jboss.org/byteman/downloads

Unzip the release and set environment variable BYTEMAN_HOME to identify the top-level directory of the resulting directory tree. It should contain several subdirectories including lib and bin.

JBossDeveloper

BMUnit : Using Byteman with JUnit or TestNG from maven and ant

Let's assume you know what Byteman is . . .

What is BMUnit?

Is There a Simple Example of How To Use BMUnit?

How Do I Inject Trace Into JUnit Tests?

The Check Rule

Using BMUnitRunner to Inject the Rules

Configuring your test to run with JUnit5

How Can I Check My Rules Are Being Run

How Do I Configure Injection Into JUnit With Maven?

Configuring maven to use JUnit5

Recommended Surefire Settings

How Do I Configure Injection Into JUnit With Ant?

Configuring ant to use JUnit5

How Do I Inject Faults Into JUnit Tests?

Using BMUnitRunner to Inject the Fault Rules

How Do I Inject Rules Into TestNG Tests?

Using BMNGRunner to Inject the Fault Rules

How Do I Configure Injection Into TestNG With Maven?

How Do I Configure Injection Into TestNG With Ant?

Where Can I Download The Tutorial Sources?

Comments