Fault Injection Testing With Byteman

Version 34

Created by adinn on Oct 31, 2011 7:17 AM. Last modified by adinn on Jul 19, 2016 10:31 AM.

You should already be familiar with Byteman and BMUnit . . .

This is the Part Three of the Byteman tutorial series, explaining how you can use Byteman to improve the simplicity, quality and coverage of your unit, integration and system tests. Part One is a basic How-To which tells you what Byteman is and how to use it from the command line to inject side-effects into Java programs. If you are new to Byteman then you probably ought to read part one before you continue with this article. Part Two is another How-To explaining how the Byteman BMUNit package integrates Byteman with JUnit and TestNG allowing you to perform simple tracing and validation of test execution and to perform basic fault injection. If you are new to BMUnit then you probably ought to read part two before you continue with this article. If you have already used Byteman and BMUnit to construct simple fault injection-based tests then then this article will help you take the next step. It will show you how to inject and propagate faults which engineer complex test scenarios which would be difficult or, in some cases, impossible to implement without the use of a tool like Byteman.

This tutorial differs from the previous ones. Rather than being a How-To it consists of a lot more more explanation, firstly of the Byteman rules it uses to construct the test scenarios but, more significantly, of the example application code. That's necessary because it is not really possible to exhibit a meaningful, complex test scenario with a simple application. However, don't worry. The code used in this tutorial is based on a small, concise and well structured library which should not be too difficult to understand.

The library supports implementation of applications which employ a stream processing execution model (sometimes also known as a process dataflow execution model) to execute code in parallel. The library code is exercised by means of a few small example programs which serve merely to show off how it can be used without fully exercising all its potential. The tutorial also employs just a small number of test routines, again merely enough to show how Byteman can engineer some specific test scenarios rather than to provide exhaustive testing. As in the previous tutorials you will be presented with instructions enabling you to download all the code and run the applications and tests as the tutorial develops.

What is Fault Injection Testing?

Fault injection is a technique which can be used to make tests, simpler, clearer and more reliable. It involves changing the application code under test at one or more specific locations so that additional code (or, possibly, alternative code) is executed when control reaches that point during the test. By carefully selecting the side effects you introduce and by including appropriate conditions to control when they are run it is possible to simulate complex test scenarios for unusual or erroneous situations, forcing the test along the code paths meant to handle these abnormal cases. This is particularly useful when trying to test code which has timing dependencies, particularly multi-threaded code.

One advantage of fault injection is that it usually requires less effort than alternative forms of testing such as writing test harnesses or implementing mock components. It often takes only a few lines of injected code to ensure that the required test scenario is exercised. A beneficial consequence of this is that the test run mostly exercises real application code rather than code introduced just for the purpose of running the test. This limits the opportunities for the test code to introduce or mask errors or to affect memory footprint and timing.

The use of fault injection also usually means that a lot of boiler plate instrumentation code required to allow tests to set up and validate test scenarios can be omitted from the application. Instead instrumentation code can be injected only where needed for each specific test. Byteman provides the further advantage that any code it injects can bypass the normal privacy restrictions which control access to non-public methods or fields. This improves on conventional testing where validation of private behaviour or data is usually not possible without including accessor methods which then get shipped as part of the deployed application.

Fault injection testing usually relies on transforming the application offline using some sort or code rewriting scheme operating on either the source or binary files. However, with a bytecode based virtual machine like the JVM it is also possible to inject changes into test code dynamically by modifying bytecode either at load time or, possibly even, after it has been loaded. Although this sounds complicated and expensive in fact the JVM's Java agent capability makes it simple and cheap to modify application bytecode or, should you need to do so, to modify bytecode belonging to Java runtime classes. Dynamic transformation has the additional advantage that application and JVM classes can be redefined solely for the duration of a specific test, injecting specific whatever fault or instrumentation changes are needed, and then uninjecting those changes afterwards. This means that within a single test run you can redefine different aspects of the application behaviour to exercise many different test scenarios. By contrast, use of offline source or binary transformation requires either maintaining several different versions of your application built from different versions of the source or else rewriting and then rebuilding the application code or, at least, some subset of it between test runs.

Can I See a Simple Example of Fault Injection?

Part two of this tutorial series provides a couple of very simple examples of fault injection. They use Byteman to inject code which simulates failures in the Java runtime's FileStream routines. This serves to test the error handling capabilities of a small test application which writes a very basic web page. If you have not looked at that tutorial it is probably better for you to do so now before reading the rest of this tutorial.

This tutorial examines more complex uses of fault injection where reliable automated testing is almost impossible to achieve any other way. Each application uses multiple threads and links them using Java runtime classes which encapsulate the dataflows they use to share inputs and outputs. So, this means that the application is not easily able to control timing of operations or provide access to data and dataflow management. All of which makes conventional testing approaches hard to apply. The tutorial starts off by showing how to use Byteman to make control and data flow visible using tracing rules. It then continues to show how fault injection can be used to modify control flow and timing, first forcing control into exception cases and second to imposing specific ordering for certain operations in order to identify the presence of a race condition in one of the applications.

An Example Pipeline Processor Application

At the end of this article you will find details of how to download and install the application source, the library classes and test code plus the Byteman rule scripts and libraries required to test it with Byteman. The source is organised as a maven project comprising an application source sub-module and three test modules. However, the source tree also contains ant build scripts allowing you to compile and package the sources and run the tests using ant. The rest of this tutorial will guide you through the steps required to perform each of these operations.

Once you have downloaded and unzipped the application source take a look at subdirectory app. This contains the application library classes and some main program classes which exercise the library. If you want to build the library you can run command

> mvn package

. . .

in the top level directory. Alternatively, if you are using ant you can run command

> ant build

. . .

This should create a directory called target in subdirectory app in which you should find a jar called bmunit-tutorial2-app-1.0.0.jar

To run the example applications change to the app directory and run any of the main classes. For example:

> cd app

> java -classpath target/bmunit-tutorial2-app-1.0.0.jar \

org.my.app.PipelineAppMain1

The following schematic describes the pipeline employed by program PipelineAppMain1:

[source] FileSource "foo.txt"

[pipe 0] PatternReplacer "adinn" --> "msmith"

[pipe 1] TeeProcessor ---> FileSink "bar1.txt" [sink 1]

[pipe 2] PatternReplacer "[Aa]ndrew" --> "Michael"

[pipe 3] TeeProcessor ---> FileSink "bar2.txt" [sink 2]

[pipe 4] PatternReplacer "(.*)[Dd]inn(.*)" =--> "\\2Smith\\1"

[sink 0] FileSink "bar.txt"

The application constructs its pipeline using 4 of the Processor implementation classes provided by the pipeline library (in package org.my.pipeline.impl) which are shown in the diagram in blue. Each of these implementation classes is a subclass of Thread and has its own run method which is implemented as a main loop. They each represent a separate process which independently consumes a stream of input data or produces a stream of output data (both in the cases where the processor has in incoming and outgoing link). The arrows linking the Processor instances shown in the diagram in red are dataflows used to feed values from one processor to the next. Each link is implemented by means of a PipedWriter belonging to the upstream processor feeding a PipedReader belonging to the downstream processor. The labels in green identify the type of processor, either a source, which feeds data into the pipeline, a sink, which accumulates data at the end of the pipeline or a pipeline element, which reads an input stream and transforms it to produce one (or more) output streams.

In this case the pipeline starts with a single FileSource which produces a stream of text characters read from file foo.txt. The processors which terminate the pipeline are all instances of class FileSink. Each of them simply dumps its stream of incoming text characters to a file on disk, specifically to files bar.txt, bar1.txt and bar2.txt.

The processors at stages 0, 2 and 4 of the array are instances of PatternReplacer. This class reads successive lines of text from its input stream, substitutes occurrences of a matched pattern with replacement text and then writes the substituted text to its output stream. The first PatternReplacer performs a simple substitution, replacing occurrences of "adinn" with "msmith". The second PatternReplacer uses an actual regular expression as its pattern, matching occurrences of "Andrew" or "andrew". The replacement text is the simple string "Michael". The third PatternReplacer uses a multi-pattern which uses two bracketed sub-patterns to match specific elements of the input text. These elements are substituted back into the output text using the replacement forms "\\1" and "\\2". So, for example, the text " Michael Dinn" would be replaced with "Smith Michael ".

Pipeline stages 1 and 3 are instances of TeeProcessor, a class which used to create a branch (also called a Tee) in the pipeline. This processor does not change the data in its input stream. Instead it simply forwards identical copies of the input to each of its two output streams.

The application main routine creates the pipeline elements, linking them together in the correct order as it does so and then calls Thread.start to run each processor. The FileSource pushes data into the pipeline. Each intermediate processors reads incoming data, transforms it and passes it on to the next processor(s). Finally, the FileSink instances dump incoming text to disk files. Once all the threads have exited and been joined by the main thread file bar1.txt should contain a dump of the original file text with the first substitutions applied, file bar2.txt should include the first and 2nd substitutions and bar.txt should include all 3 substitutions. After running the program you can compare the files to see how each of the substitution stages has introduced changes to the text:

> diff foo.txt bar1.txt

15c15

< adinn

---

> msmith

> diff bar1.txt bar2.txt

6c6

< Andrew is a member of the Red Hat JVM development team and also

---

> Michael is a member of the Red Hat JVM development team and also

10c10

< Andrew has been writing system software for over 25 years, 10 of

---

> Michael has been writing system software for over 25 years, 10 of

17c17

< andrew dinn

---

> Michael dinn

> diff bar2.txt bar.txt

17c17

< Michael dinn

---

> Smith Michael

As you can see, specific regular expression patterns are matched and substituted with replacement text by each pipeline stage.

The other main classes you can try out are PipelineAppMain2 and PipelineAppMain3. They use a more sophisticated matching and substitution model (which will be explained later in the tutorial) to perform matches and substitutions across multiple lines. These latter classes use String inputs and generate String outputs so the main routines can print the input, intermediate results and output to System.out. You can browse these classes now to understand what they do or read the rest of this tutorial which will gradually explain their behaviour.

A Very Testing Problem

The pipeline library was chosen for this tutorial because it provides some interesting opportunities for timing and communication bugs to arise and it is very hard to debug or test using conventional means. Each stage of an application pipeline is a separate thread and there is no guarantee that operations performed by one element of the pipeline will occur before or after operations which occur in a downstream pipeline element. Obviously, a processor cannot transform a line of input until the upstream processor has made it available. However, there is no guarantee that a given line of input will be processed by a downstream processor either before or after a subsequent line has been processed and made available by the upstream processor. With a long pipeline operations can happen in many different orders. That's ok if everything is correct but it makes it very difficult to see what is wrong when there is a bug, especially as some of those ordering may only turn up rarely, usually during deployment on heavily loaded machines.

Another difficulty when it comes to testing this code is that data is passed from thread to thread using a pair of PipedWriter and PipedReader instances. These Java runtime classes completely encapsulate the data transfer between the different threads in the application. They provide no way of identifying what data is buffered in the pipeline at any given point during execution nor of controlling how or when it is propagated from writer to reader. This makes it very difficult to control the progress of the test ensuring that specific ordering which may occur at deployment time are actually realised and validated during testing.

Even if you were to give up on automated testing and tried to use a debugger to verify the correct operation of this application it would be very difficult to ensure that everything operated correctly and in all possible orderings. If you try stop one thread to see what it was doing then while you are stepping through the operations of a particular stage all the rest of the upstream pipeline threads and whatever threads in the downstream pipeline still had data to process would continue to run. What is more when several threads are running the same code breaking a specific method means you usually end up having to step through several threads at the same time.

The implementation of the pipeline classes could help with the task of exercising different orderings and validating correct operation by providing an execution mode which supported inspecting data as it is processed and/or sequencing execution of pipeline elements. However, that is unattractive for two reasons. Firstly, the main attraction of using the process dataflow model is to try to obtain performance gains by executing operations in parallel. Adding support for serial execution is hard to do without slowing down the application when serial execution is switched off. Even if it such a slowdown can be avoided the presence of a serial vs parallel execution mode is likely to change globally the way the application runs and therefore make tests less realistic. A specific, limited modification to execution which only controls the timing of operations key to a given a test scenario would be better as it would still run most of the application code under the conditions which will be observed at deploy time.

The second problem is that provision of code hooks which allow, say, serialisation of execution or exposure of data which is supposed to be private to the application makes the deployed code bulkier, more complex and less secure than it really needs to be. This can have an impact on performance and may well also affect re-usability and maintainability. Injecting code at test time to provide only the limited control or data accessibility needed for a given test scenario avoids these risks.

The Pipeline Core Classes

In order to show how we can test the pipeline library it is first necessary to understand a little about how it is implemented. The library is split into two packages, core and impl, and we will start by taking a look at the core classes. Once they are understood it will be easy to follow how the classes in the impl package have been built and, indeed, to extend the library with new impl classes.

The class and interface hierarchy for the core classes is a good place to start. Here is a simple schematic for the main classes and interfaces:

.------------. .-----------------------.

| Source | +--------+ | Sink |

'------------' +----+---+ '-----------------------'

\ \ | / /

\ +----------------------+---------------------+ /

\ | \____________ | ______/ | /

+-------\--+---------+ +---\----+-----/----+ +-------+--/--------+

| run() | | run() | | run() |

+--------------------+ +--------+----------+ +-------------------+

+--------+----------+

| TextLineProcessor |

| transform(String) |

+-------------------+

Java class names are in brown and the extends relationship between them is shown by red links. Java interface names are in green and the implements links between them and the classes are shown by blue links. The most important fields and methods belonging to each class/interface are listed below each name.

Interfaces Source and Sink define a protocol which is used to connect an upstream Source to a downstream Sink and this protocol is relied upon at create time. Pipelines elements are constructed in the order of dataflow. So, when a Sink is created it's constructor takes as an argument the upstream Source which feeds it. The required step at this point is for the upstream Source to establish a link to its downstream Sink. The protocol expects the constructor for the Sink to call Source.feed(this) passing itself as argument. The Source creates a PipedWriter which it stores locally in field output. It connects this to a PipedReader and makes a call back to Sink.setInput(pipedReader) thereby passing the downstream Sink a handle on the PipedReader. The Sink stores this reader in field input. So, the Source is now able to write data to output which the Sink can read from input.

The three main classes in the library are abstract classes which implement either the Source or Sink protocol. Class PipelineProcessor implements both protocols because it abstracts the behaviour needed both to accept input and generate output. All three of these subclasses extend Thread, providing their own implementation of method run(). So, these abstract classes encapsulate the work of creating the input and output links, handling of IO Exceptions and scheduling the processing of input and output. They delegate the actual reading and/or writing of input and/or output data to an abstract method which their subclasses must define.

As an example, here is some of the source code for class SourceProcessor:

public class SourceProcessor extends thread implements Source {

PipedWriter output;

public feed(Sink sink) throws IOException {

if (output != null) {

throw new IOException("output already connected");

}

output = new PipedWriter();

sink.setInput(new PipedReader(output));

}

public abstract void produce() throws IOException;

public void run() {

if (output==null) { //nothing to do

return;

}

try {

produce();

} catch (IOException ioe) {

ioe.printStackTrace();

} finally {

try {

output.close();

} catch (IOException ioe) {

ioe.printStackTrace();

}

. . .

A subclass of SourceProcessor only has to implement produce, writing data to output and then closing output when there are no more items to post. One example of a SourceProcessor implementation is class CharSequenceSink which writes data from a CharSequence (that's an interface implemented by, amongst other things, class String).

public class CharSequenceSource extends SourceProcessor

{

CharSequence charseq;

public CharSequenceSource(CharSequence charseq) throws IOException {

super();

this.charseq = charseq;

}

@Override public void produce() throws IOException {

if (charseq instanceof String) {

output.write((String) charseq);

} else {

int l = charseq.length();

for (int i = 0; i < l; i++) {

output.write(charseq.charAt(i));

}

The other two abstract processor classes operate similarly. SinkProcessor.run delegates to abstract method consume and PipelineProcessor.run delegates to abstract method processPipeline:

public class SinkProcessor extends Thread implements Sink {

. . .

public void run()

{

if (input==null) { //nothing to do

return;

}

try {

consume();

} catch (IOException ioe) {

ioe.printStackTrace();

} finally {

try {

input.close();

} catch (IOException ioe) {

ioe.printStackTrace();

}

. . .

public class PipelineProcessor extends Thread implements Source, Sink {

. . .

public void run() {

boolean excepted = false;

if (input == null || output == null) {

throw new RuntimeException("unconnected pipeline");

}

try {

processPipeline();

} catch (IOException ioe) {

//ioe.printStackTrace();

// when we got an exception then we need try to close our

// input as well as our output otherwise we may leave our

// feeder thread sitting on a write to a full pipeline.

// so remember that this happened

excepted = true;

} finally {

try {

output.close();

} catch (IOException ioe) {

// only print this if we have not already printed another

// one. if we already excepted it might have been caused

// by the same stream

if (!excepted) {

// ioe.printStackTrace();

}

if (excepted) {

try {

input.close();

} catch (IOException ioe2) {

// the input may be the source of the original exception

//so don't bother to print this

}

. . .

PipelineProcessor Core SubClasses

The library provides two more core classes which extend base class PipelineProcessor. The first is class TeeProcessor which is used to split the input pipeline into two streams:

public class TeeProcessor extends PipelineProcessor {

protected PipedWriter output2;

public TeeProcessor(Source source) throws IOException {

super(source);

this.output2 = null;

}

/**

* TeeProcessor expects and will only accept two feed requests.

* @param sink

* @throws IOException

public void feed(Sink sink) throws IOException

{

if (output == null) {

super.feed(sink);

} else if (output2 == null) {

output2 = new PipedWriter();

sink.setInput(new PipedReader(output2));

} else {

throw new IOException("output already connected");

}

/**

* Copies the input stream to both output streams

* @throws RuntimeException if a second output has not been configured

public void processPipeline() throws IOException

{

if (output2 == null) {

throw new IOException("unconnected tee");

}

try {

int next = input.read();

while (next != -1) {

output.write(next);

output2.write(next);

next = input.read();

}

} finally {

output2.close();

}

PipelineProcessor is also extended by another abstract class located in the core package, TextLineProcessor . This subclass imposes a line discipline on its input text, providing an implementation of method processPipeline() which loops, repeatedly reading a line from the input text, processing it and then appending the (possibly) transformed version of the line to the output. It delegates the actual processing to its subclasses which are expected to implement abstract method transform(String). This method takes the input text (omitting end of line characters) as argument and returns a transformed line (or the original input if it is acceptable) as the value to be written to the output.

public abstract class TextLineProcessor extends PipelineProcessor {

. . .

public void processPipeline() throws IOException

{

TextLine lineBuffer = new TextLine(input);

String text = lineBuffer.readText();

while (text != null) {

text = transform(text);

output.write(text);

if (lineBuffer.isCrLf()) {

output.write('\r');

output.write('\n');

} else if (lineBuffer.isLf()){

output.write('\n');

}

text = lineBuffer.readText();

}

public abstract String transform(String line);

. . .

Note that class TextLineProcessor relies on an auxiliary class TextLine to handle parsing of the input text and retain details of the end of line convention.

Non-Core Library Classes

All the remaining subclasses of PipelineProcessor provided by the library belong to the impl package and are subclasses of TextLineProcessor. PatternReplacer, the class we saw earlier in the implementation of PipelineAppMain1 is one such subclass. This abstraction of much of the necessary behaviour in the core classes makes it easy to extend the library to add new pipeline processor elements. Class PatternReplacer does not need to know how to connect itself into a pipeline, nor how to read or write lines of text to input streams or from output streams, nor does it need to provide code to handle IO exceptions. All it does is implement method transform, accepting a String as input, looking for pattern matches and returning a substituted String if matches are found or the original String if there is no match.

Similarly, the input classes FileSource and CharSequenceSource which fill the pipeline from a disk file or a String (actually, any implementation of interface CharSequence will do) are also very simple. merely needing to concentrate on implementing method produce. Likewise, the output classes FileSink and CharSequence merely need to implement method consume, dumping input data to a file or collecting it in memory using, say, a StringBuffer. As an example of a SinkProcessor implementation here is the code for CharSequenceSink. Notice that the implementation of produce is only a few lines. The bulk of the code is there to implement interface CharSequence, allowing the text collected by the sink to be retrieved once processing has completed.

public class CharSequenceSink extends SinkProcessor implements CharSequence {

private StringBuffer buffer;

public CharSequenceSink(Source source) throws IOException {

super(source);

this.buffer = new StringBuffer();

}

public void consume() throws IOException {

int next = input.read();

while (next >= 0) {

buffer.append((char) next);

next = input.read();

}

public int length() {

return buffer.length();

}

public char charAt(int index) {

return buffer.charAt(index);

}

public CharSequence subSequence(int start, int end) {http://github.com/adinn/bmunit-tutorial2/blob/master/app/src/main/java/org/my/pipeline/impl/PatternReplacer.java

return buffer.subSequence(start, end);

}

public String toString()

{

return buffer.toString();

}

A Simple Test of the Pipeline Library Functionality

The sample application contains a subdirectory called junit, which includes a single test class BytemanJUnitTests. It contains one test method, testPipeline, which exercises a pipeline built using PatternReplacer. Here is how the method creates its pipeline:

@Test public void testPipeline() throws Exception

{

System.out.println("testPipeLine:");

String input = "hello world! goodbye cruel world, goodbye!\n";

CharSequenceSource cssource = new CharSequenceSource(input);

PatternReplacer replacer = new PatternReplacer("world", "mum", cssource);

CharSequenceSink cssink = new CharSequenceSink(replacer);

. . .

In this example the pattern is a simple String "world" and the replacement is another String "mum". These values are passed to the PatternReplacer in the constructor. The input text is fed into the pipeline by a CharSequenceSource which writes characters from a constant String downstream to the next pipeline processor in line. Notice how the CharSequenceSource is passed as an argument in the constructor for the PatternReplacer so that the two pipeline elements can be connected. The output text is collected by a CharSequenceSink which collects incoming data as a sequence of characters. It is constructed with the PatternReplacer as its upstream Source. Once the pipeline has been started and has completed processing the output text collected by the CharSequenceSink can be retrieved by calling its toString() method.

The test method continues as follows

. . .

cssource.start();

replacer.start();

cssink.start();

cssource.join();

replacer.join();

cssink.join();

String output = cssink.toString();

assert(output.equals("hello mum! goodbye cruel mum, goodbye!\n"));

}

The test routine must wait for all three pipeline stages to complete before it can access the output text. The assertion ensures that the word "world" has been replaced at every occurrence with the word "mum". So, clearly this test is complete. Hiowever, even if it passes this is not very interesting it is not clear what the test is actually doing. What is worse, if it fails we will have no idea of where it is going wrong.

How Can We Trace Pipeline Activity?

It would be nice to be able to view what is happening when this test gets run so we can see when a line gets transformed. We could insert some TeeProcessor stages into the pipeline but that is problematic on several counts. Firstly, it means we are testing a very different pipeline. Introducing extra pipeline stages will certainly affect the timing of the other pipeline elements and it also means that the stages we are interested in are not directly feeding each other as they would in a real application built using the library.

Secondly, adding TeeProcessor stages makes the test code more complicated to follow by introducing elements into the test which are not really necessary for exercising the code we are interested in. This code is only present in order to provide a means of monitoring execution during the test. It would be nice if we could keep this instrumentation code separate from the code driving the test.

Thirdly, the TeeProcessor stages can show us what goes into each pipeline stage or what comes out but they cannot show us any intermediate state to explain what is happening as each pipeline element processes its input. In fact, with this application the problem is particularly difficult. Since a pipeline processor is implemented as a self-contained thread the test thread cannot reliably check the state of fields in the pipeline objects at different stages during execution.That would require them to provide some sort of synchronization mechanism so that the test code could stop them executing, identify their state and then allow them to continue.

These three issues are not just specific to this example. They are typical of almost every application under test. Luckily, we can use some Byteman to address all three of these problems. We can inject code which observes specific behaviours of the source, sink and pattern replacer threads in a configuration much closer to the one we really wish to test with no need to clutter our test code or, indeed application code, with instrumentation routines.

The trace rules we will use are contained in script trace.btm in directory junit/src/resources:

# A rule which dumps the text passed to a CharSequenceSource when it is created

RULE trace input characters

CLASS CharSequenceSource

METHOD <init>(CharSequence)

AT EXIT

IF TRUE

DO traceln("#INPUT\n" + $1.toString() + "#END")

ENDRULE

This first rule dumps the input text supplied when a CharSequenceSource is created. The text is bracketed with #INPUT and #END, allowing us to see what is being fed into the pipeline.

# A rule which dumps the text stored in a CharSequenceSink

# when it is retrieved

RULE trace output characters

CLASS CharSequenceSink

METHOD toString()

AT EXIT

IF TRUE

DO traceln("#OUTPUT\n" + $! + "#END")

ENDRULE

This second rule dumps the text collected by the CharSequenceSink bracketed with #OUTPUT and #END when it is retrieved by the test routine, allowing us to see what emerged from the pipeline. Note that the rule is triggered at the end of the call to method toString(). It refers to the value being returned form the call using special variable $!.

# A rule which traces replacements made by a PatternReplacer

# if the returned line of text differs from the input line it

# is dumped to System.out

RULE dump pattern transformed text

CLASS PatternReplacer

METHOD transform(String)

AT EXIT

IF NOT $1.equals($!)

DO traceln(" patternReplace(" + $1 + " -> " +$! +")")

ENDRULE

This rule is triggered at the end of the call to method transform(String) of class PatternReplacer. The condition compares the text line provided as argument, $1, with the return value, $!. If these two strings differ then then the rule fires and prints a trace message. If the lines are the same it does nothing.

Finally we add some rules which allow us to see pipeline threads starting and exiting

# A rule which prints a trace message whenever a thread's run method is entered

RULE trace Thread.run entry

CLASS ^Thread

METHOD run()

AT ENTRY

IF TRUE

DO traceln("entered run for " + $this.getClass().getName());

ENDRULE

# A rule which prints a trace message whenever a thread's run method is exited

RULE trace Thread.run exit

CLASS ^Thread

METHOD run()

AT EXIT

IF TRUE

DO traceln("exited run for " + $this.getClass().getName());

ENDRULE

Note that the ^ before the class name means that these rules are injected through class Thread into the run method of all subclasses, including the core pipeline processor classes..

Using BMUnit to Inject the Trace Rules

The rules in script trace.btm can be loaded automatically when the test is run by attaching a few annotations to the test class:

@RunWith(BMUnitRunner.class)

@BMUnitConfig(loadDirectory="target/test-classes")

@BMScript(value="trace")

public class BytemanJUnitTests

{

. . .

The first @RunWith annotation tells JUnit to use the Byteman class BMUnitRunner to run the tests. BMUnitRunner is a subclass of the standard JUnit4 test runner class and it delegates most of the job of running the test to JUnit. However, it operates as a wrapper class loading the Byteman agent when testing begins and loading and unloading Byteman rules as they are needed to run specific tests.

In this case the runner detects the presence of the @BMUnitConfig and @BMScript annotations on the test class. The @BMUnitConfig annotation sets one of the configuration parameters which controls operationof the BMUNit package, specifically it tells BMUNit that Byteman scritps should be looked for in directory "target/test-classes" which is were maven copies the scripts located in the test/resources directory. The @BMScript annotation identifies a script to be loaded for the duration of this test.

Before running any of the test methods in the class BMUnitRunner ensures the Byteman agent has been loaded into the current JVM. Next it uploads the rules in script trace.btm via the agent's listener socket. Once all tests in the class have been executed BMUnitRunner connects to the agent listener and unloads all the rules defined in the script.

If you are using maven then you can run the tests in module junit from the root directory of your download tree by enabling the junit profile. The profile entry for profile junit in the top-level pom adds subdirectory junit as a module, ensuring that its tests get run as part of the top-level build process

> mvn test -P junit

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running org.my.BytemanJUnitTests

byteman jar is /ssd/home/adinn/.m2/repository/org/jboss/byteman/byteman/2.2.1/byteman-2.2.1.jar

Setting org.jboss.byteman.allow.config.update=true

testPipeLine:

#INPUT

hello world!

goodbye cruel world!

goodbye!

#END

entered run for org.my.pipeline.impl.PatternReplacer

entered run for org.my.pipeline.impl.CharSequenceSource

entered run for org.my.pipeline.impl.CharSequenceSink

exited run for org.my.pipeline.impl.CharSequenceSource

patternReplace(hello world! -> hello mum!)

patternReplace(goodbye cruel world! -> goodbye cruel mum!)

exited run for org.my.pipeline.impl.CharSequenceSink

exited run for org.my.pipeline.impl.PatternReplacer

#OUTPUT

hello mum!

goodbye cruel mum!

goodbye!

#END

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.427 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

. . .

As you can see the Byteman rules are injected and display the input, the replacement operation and the output.

If you are using ant then you can run the test from the top level directory with the following command

> ant junit

. . .

test:

[copy] Copying 1 file to /ssd/home/adinn/jboss/byteman/git/tutorial2/junit/target/test-classes

[junit] testPipeLine:

[junit] #INPUT

[junit] hello world! goodbye cruel world, goodbye!

[junit] #END

[junit] patternReplace(hello world! goodbye cruel world, goodbye! -> hello mum! goodbye cruel mum, goodbye!)

[junit] #OUTPUT

[junit] hello mum! goodbye cruel mum, goodbye!

[junit] #END

. . .

Note that for this ant command to work you must ensure that BYTEMAN_HOME is set and identifies the directory where you downloaded and unzipped the Byteman binary release. Also, make sure you have run command ant build first to ensure that the application classes have been compiled and installed in the application jar.

Breaking the Pipeline Application Using Byteman

Now let's consider some Byteman rules which test class TextLineProcessor by injecting fault code, in this case exercising the application exception handling. Injecting faults is done in exacttly the same way as you inject trace rules i.e. using annotations provided by the BMUnit package. However, in most cases you want fault rules to be injected before a specific test and then removed after the test has completed. This is because you normally only want to break the application or Java runtime in a way that is highly specific to the test in question.

The junit2 subdirectory contains two further JUnit tests in class BytemanJUnitTests2 which employ rules to break the application. Here is the first method:

@RunWith(BMUnitRunner.class)

@BMScript(value="trace", dir="target/test-classes")

public class BytemanJUnitTests2

{

@Test

@BMRule(name="throw IOException at 1st transform",

targetClass = "TextLineProcessor",

targetMethod = "processPipeline",

action = "throw new java.io.IOException()")

public void testErrorInPipeline() throws Exception

{

System.out.println("testErrorInPipeline:");

StringBuffer buffer = new StringBuffer("hello world!");

buffer.append(" goodbye cruel world, goodbye!\n");

CharSequenceSource cssource = new CharSequenceSource(buffer);

PatternReplacer replacer = new PatternReplacer("world", "mum",reader);

CharSequenceSink cssink = new CharSequenceSink(replacer);

cssource.start();

replacer.start();

cssink.start();

cssource.join();

replacer.join();

cssink.join();

String output = cssink.toString();

assert(output.equals(""));

}

. . .

If you ignore the different assertion this is essentially the same method as in the previous example. However, this test employs an @BMRule annotation on the first test method which injects an exception into the pipeline. By placing the annotation on the method this ensures that the rule is loaded just before running the test and is unloaded just after the test completes. So, this stops the rule affecting the next test method.

In contrast to the @BMScript annotation on the class, @BMRule provides the rule text inline in the annotation. The rule is injected into method processPipeline of class TextLineProcessor and it throws an IOException when it is fired. The location field of the annotation is left unspecified so it defaults to "AT ENTRY". Similarly, the condition defaults to "TRUE". So, this rule will cause the PatternReplacer to throw an exception as soon as it starts running. The output stream to the sink should be closed with no input having been written. The assert condition checks that the output is indeed an empty String.

Notice that the class still has the @BMScript annotation to load the trace rules. These rules are loaded before any tests in the class are run so when this test method and the next test method are executed they will still generate trace showing the input, text replacements and output. You can use either annotation, @BMScript or @BMRule, at the class or method level but this configuration where you have trace rules in a script attached to the class and inline fault rules attached to the test method is the most common one.

The second test method is a variant on the method above but it uses a pair of countdown rules to delay the exception until after some data has passed through the pipeline.

@Test

@BMRules(rules={@BMRule(name="create countDown for TextLineProcessor",

targetClass = "TextLineProcessor",

targetMethod = "<init>",

action = "createCountDown($0, 2)"),

@BMRule(name="throw IOException at 3rd transform",

targetClass = "TextLineProcessor",

targetMethod = "processPipeline",

targetLocation = "AT CALL transform(String)",

condition = "countDown($0)",

action = "throw new java.io.IOException()")})

public void testErrorInStuffedPipeline() throws Exception

{

System.out.println("testErrorInStuffedPipeline:");

StringBuffer buffer = new StringBuffer("hello world!\n");

buffer.append("goodbye cruel world, goodbye!\n");

for (int i = 0; i < 40; i++) {

buffer.append("goodbye! goodbye! goodbye!\n");

}

CharSequenceSource cssource = new CharSequenceSource(buffer);

PatternReplacer replacer = new PatternReplacer("world", "mum",reader);

CharSequenceSink cssink = new CharSequenceSink(replacer);

cssource.start();

replacer.start();

cssink.start();

cssource.join();

replacer.join();

cssink.join();

String output = cssink.toString();

assert(output.equals("hello mum!\ngoodbye cruel mum, goodbye!\n"));

}

Notice that we are using more than one @BMRule annotation here so we use an enclosing @BMRules annotation to bracket them. This is ok when you have two or three simple rules but for larger numbers it is usually clearer to load your rules from a script. The equivalent script would be as follows

RULE create countdown for TextLineProcessor

CLASS TextLineProcessor

METHOD <init>

IF TRUE

DO createCountDown($0, 2)

ENDRULE

RULE throw IOException at 3rd transform

CLASS TextLineProcessor

METHOD processPipeline

AT CALL transform(String)

IF countDown($0)

DO THROW new java.io.IOException()

ENDRULE

The first rule is attached to the constructor for TextLineProcessor (<init> is the JVM internal name for a constructor). It will be triggered whenever a TextLineProcessor is constructed. It calls built-in operation createCountDown(Object, int) to create a CountDown instance with counter 2. The first argument, the TextLineProcessor instance $0, is used to label the CountDown, allowing the same CountDown to be retrieved by later rules. As the name suggests this CountDown object is used to ensure that a rule fires when a counting process reaches zero.

The second rule is injected into the loop in method processPipeline() just before the call to method transform processes the next line of text. The condition employs the related built-in countDown(Object), passing the TextLineProcessor object $0 to identify the CountDown to use. This built-in only returns true when the CountDown counter has value 0. So, at the first call to transform(String) the counter decrements from 2 to 1 and the condition evaluates to false. This means the rule does not fire and execution returns to method processPipeline() continuing with the call. At the second call the counter decrements from 1 to 0 and the condition is still false so transform(String) is called again. At the third call the counter is 0 so the call to countDown(Object) returns true. The rule fires and throws an IOException, causing the TextLineProcessor to close its input and output streams and exit.

There is another detail here worth considering. For this test the CharSequenceSource posts 40 extra lines into its output. This is enough to fill the piped stream feeding the PatternReplacer, causing the CharSequenceSource to suspend. So, when the exception is thrown the CharSequenceSource should wake up under a suspended write operation and print the resulting pipe full exception to System.err. In the previous test an exception (normally) is not printed because the CharSequenceSource has already written its two lines of input text into the ouptut pipe and exited before the PatternReplacer starts running.

Running the Fault Injection Code

The tests in directory junit2 can be run by executing the following command

> mvn test -P junit2

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running org.my.BytemanJUnitTests2

byteman jar is /ssd/home/adinn/.m2/repository/org/jboss/byteman/byteman/2.2.1/byteman-2.2.1.jar

Setting org.jboss.byteman.allow.config.update=true

testErrorInPipeline:

#INPUT

hello world! goodbye cruel world!

#END

#OUTPUT

#END

testErrorInFullPipeline:

#INPUT

hello world!

goodbye cruel world!

goodbye! goodbye! goodbye!

. . .

goodbye! goodbye! goodbye!

#END

patternReplace(hello world! -> hello mum!)

patternReplace(goodbye cruel world! -> goodbye cruel mum!)

*** IO exception created here ***

java.io.IOException.<init>(IOException.java:59)

java.io.PipedReader.receive(PipedReader.java:173)

java.io.PipedWriter.write(PipedWriter.java:124)

org.my.pipeline.impl.CharSequenceSource.produce(CharSequenceSource.java:52)

org.my.pipeline.core.SourceProcessor.run(SourceProcessor.java:70)

***

java.io.IOException: Pipe closed

at java.io.PipedReader.receive(PipedReader.java:173)

at java.io.PipedWriter.write(PipedWriter.java:124)

at org.my.pipeline.impl.CharSequenceSource.produce(CharSequenceSource.java:52)

at org.my.pipeline.core.SourceProcessor.run(SourceProcessor.java:70)

#OUTPUT

hello mum!

goodbye cruel mum!

#END

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.414 sec

. . .

If you are using ant you can execute the JUnit fault injection tests from the top level directory wiht the following command

> ant junit2

. . .

You should see essentially the same output as when you run the maven version of the test.

Breaking and Entering a Timing Window

The BMUnit package can be used with TestNG in almost exactly the same way as it is used with JUnit. The only major difference is in how you inform TestNG that the test requires the use of Byteman rules. We saw that with JUnit you need to add an @RunWith annotation to your test class. For TestMG based tests you need to make your test class inherit from a class called BMNGRunner. This parent class provides all the behaviour needed to ensure that rules are loaded before running tests and unloaded after tests complete.

The next example based on TestNG shows some rules which display a timing dependency in the pipeline application. By introducing synchronizations into the threads in the pipeline it is possible to control the order in which specific lines are processed by different pipeline stages. This can be used to prove that given the same pipeline and input the output may vary depending upon the order in which the threads execute. This timing dependency is not normally apparent because the upstream thread in the pipeline tends to process the critical line before the downstream thread starts executing. However by injecting suitable rules we can delay the upstream thread ensuring that the downstream thread runs first or vice versa.

The ability to force threads into uncommon timing windows is one of the most powerful capabilities provided by fault injection testing. It makes it simple to exercise code reliably and repeatably in circumstances which happen only rarely when the application is deployed. The technique displayed here is applied to a relatively simple and slightly artificial case where there is a clear (and deliberate!) error in the design of the library. However, variations on this technique have proved invaluable for confirming the correctness of, amongst other applications, many of the multi-threaded libraries which go to make up the JBoss application server.

The timing test employs two new subclasses of TextLineProcessor, Binder and BindingReplacer. A Binder passes input lines straight through to its output without modification. However, it does process the input. The Binder matches elements out of input lines and associates the text with a variable, storing the association, for preference referred to as a binding, in a BindingMap. So, for example, a Binder configured to use pattern "[Aa] ([A-Za-z]+)" and prefix "X" would match the input line "A boy and a dog" twice, binding variable X1 to "boy" and variable X2 to "dog". Note that this example match expression uses a group term (identified by a pair of round brackets) to identify a substring of the matched String to be used as the bound value. If no group term is included then the whole matched value is used.

BindingReplacer works in tandem with Binder to insert bound values into text. It uses a BindingMap to lookup values for bound variables and substitutes references to the variables with the relevant values. So, given the input line "The ${X1} chases the ${X2}" and the bindings [X1 -> "boy", X2 -> "dog"] a BindingReplacer would output the transfomed line "The boy chases the dog".

The test class BytemanNGTests is located under directory testng. It implements two tests both of which employ the same pipeline and input setup. Here is the one which displays the normal execution order

@BMScripts(scripts = {@BMScript(value="trace", dir="target/test-classes"),

@BMScript(value="trace2", dir="target/test-classes")})

public class BytemanNGTests extends BMNGRunner

{

@BMScript(value="timing", dir="target/test-classes")

@Test

public void testForwardPublishingGood() throws Exception

{

System.out.println("testForwardPublishingGood:");

BindingMap bindings = new BindingMap();

StringBuffer buffer = new StringBuffer("the boy threw the stick for the dog to catch\n");

buffer.append("a ${X1} broke a ${X4} with a ${X2}\n");

buffer.append("the boy threw the stick at the window\n");

CharSequenceSource cssource = new CharSequenceSource(buffer);

Binder binder = new Binder("the ([A-Za-z]+)", "X", bindings, reader);

BindingReplacer replacer = new BindingReplacer(bindings, binder);

CharSequenceSink cssink = new CharSequenceSink(replacer);

cssource.start();

binder.start();

replacer.start();

cssink.start();

. . .

The class is annotated with an @BMScripts annotation used to bracket a pair of @BMScript annotations. The first of these annotations loads the trace script trace.btm used in the previous examples.The second one loads an extra trace script trace2.btm which includes rules displaying the operation of the Binder, BindingReplacer and BindingMap.

The test pipeline comprises a Source, a Binder, a BindingReplacer and a Sink. Notice that the binding map created at the start of the method is passed to both the Binder and the BindingReplacer. So, bindings inserted by the Binder can be used by the BindingReplacer.

The first input line contains three terms matching the Binder pattern "the ([A-Za-z]+)": "the boy", "the stick" and "the dog". So, when the Binder sees these values it binds the successive variables X1, X2 and X3 to each respective match updating the binding map to [X1 -> "boy", X2 -> "stick", X3 -> "dog"]. It then passes the line on to the BindingReplacer unchanged. The second line of input does not contain any matching terms so this line is just passed straight through. The third line contains two matches, "the boy" and "the stick", whose matched value is already bound, to variables X1 and X2, respectively. So, these matches do not result in any new bindings. However, the match for "the window" is new so variable X4 is bound to "window". The bindings are updated to [X1 -> "boy", X2 -> "stick", X3 -> "dog", X4 -> "window"] and line 3 is passed on to the BindingReplacer.

The BindingReplacer is not interested in lines 1 and 3 because they do not contain terms of the form ${AANN}. However, it does attempt to transform line 2, substituting values for variables X1, X2 and X4. It should begin to be clear now where the timing problem will occur. Normally, the Binder will process lines 1 2 and 3 before the BindingReplacer is able to process line 2. That means that all four variables will be bound before the BindingReplacer tries to look them up. However, if by some chance the Binder thread is suspended after writing line 2 and the BindingReplacer thread is allowed to run and process line 1 and line 2 then the binding for X4 will not be present in the binding map.

So, how can we ensure that a specific order of execution is followed? Well, the critical issue is to control whether or not the Binder processes its third line before the BindingReplacer processes its second line. The first thing we need to be able to do is count which line each of these pipeline elements is processing. Then we need to be able to stop each thread just before it processes the relevant line and then synchronize with it again just after so we can be sure the line has been dealt with. The rules which do this are contained in script timing.btm in subdirectory testng/src/test/resources. Here are the first two rules

RULE create Binder rendezvous and counter

CLASS Binder

METHOD <init>

IF TRUE

DO createRendezvous($0, 2, true);

createCounter($0)

ENDRULE

RULE create BindingReplacer rendezvous and counter

CLASS BindingReplacer

METHOD <init>

IF TRUE

DO createRendezvous($0, 2, true);

createCounter($0)

ENDRULE

Each of these rules calls built-in createRendezvous(Object, int, boolean) to create an object called a Rendezvous (otherwise known as a barrier or thread barrier). The first argument to the createRendezvous call, used to label the Rendezvous so it can be looked up later, is $0, the Binder or BindingReplacer, respectively. So, each of the pipeline elements has its own personal Rendezvous.

A Rendezvous is created with a specific count supplied as the second argument, in this case 2, which identifies the number of threads which will meet at the rendezvous. When the Binder or BindingReplacer thread executes one of the rules below which calls the related built-in rendezvous(Object) it may suspend before returning from the call. The rule is that no thread can exit from a call to rendezvous until N threads have entered the Rendezvous where N is the count supplied when the Rendezvous is created. So, in this case when either the Binder or the BindingReplacer thread calls rendezvous it must synchronize with another thread which has also called rendezvous before it can continue. In either case, the other thread is going to be the one running the test. By deciding which order the test thread enters each of these Rendezvous it can control the order in which the Binder and BindingReplacer threads execute. Note that a Rendezvous itself is not ordered -- it doesn't matter whether, say, the test thread or the Binder thread arrives first, just that both threads must be inside a call to rendezvous before either can exit.

The third, optional argument to createRendezvous determines whether the rendezvous is deleted at the first exit (the default) or is reset to allow repeated rendezvous operations. For this test we need each thread to enter the rendezvous twice so the argument true is supplied to allow the rendezvous to be re-entered.

The second call in the DO list uses built-in createCounter to create a numeric counter. Each counter is labelled using the Binder or BindingReplacer instance, respectively, allowing it to be identified. up in subsequent calls. Counters are normally used to maintain running totals as a program progresses and Byteman provides built-in operations allowing a Counter to be incremented, decremented or read from injected code. In this case the counters are used to track how many input lines have been read.

The next rule is used to count lines as they are processed by the Binder and BindingReplacer. This makes it possible to decide when to enter the endezvous

RULE increment Binder or BindingReplacer line counter

CLASS ^TextLineProcessor

METHOD transform

AT ENTRY

IF TRUE

DO incrementCounter($0)

ENDRULE

This rule is injected into the transform method declared by TextLineProcessor. However, this is an abstract method. The symbol ^ in front of the class name requests that the rule is injected down the class hierarchy into any implementation defined by a subclass of TextLineProcessor. So, it gets injected into the methods defined by both Binder and BindingReplacer and whenever a new line of text is presented to either pipeline element its related Counter gets incremented. Notice how $0 is passed as argument to identify the correct Counter instance.

The next two rules are used make the Binder enter a rendezvous just before it processes the 3rd line of text and just after it has completed processing the line, respectively

RULE enter first Binder rendezvous

CLASS Binder

METHOD transform

AT ENTRY

IF readCounter($0) == 3

DO rendezvous($0)

ENDRULE

RULE enter second Binder rendezvous

CLASS Binder

METHOD transform

AT EXIT

IF readCounter($0) == 3

DO rendezvous($0)

ENDRULE

The first rule is injected AT ENTRY so it will be triggered before the line is processed. It only fires when the counter is 3 so the Binder thread will enter the rendezvous for the first time just before it processes the 3rd line. SImilarly, the second rule is injected AT EXIT. So, the Binder thread will enter the rendezvous again just after it has processed the 3rd line.

The rules for the BindingReplacer are much the same except that the counter is compared to constant value 2.

RULE enter first BindingReplacer rendezvous

CLASS BindingReplacer

METHOD transform

AT ENTRY

IF readCounter($0) == 2

DO rendezvous($0)

ENDRULE

RULE enter second BindingReplacer rendezvous

CLASS BindingReplacer

METHOD transform

AT EXIT

IF readCounter($0) == 2

DO rendezvous($0)

ENDRULE

The one remaining piece of the puzzle is how the test thread is going to enter these rendezvous? After all the test code does not have access to Byteman built-in operations. The answer is that we arrange for the test code to call a dummy method and inject a call to rendezvous into this dummy method. Here is the rule

RULE allow test thread to enter rendezvous

CLASS BytemanNGTests

METHOD triggerRendezvous

IF TRUE

DO rendezvous($1)

ENDRULE

The rule is attached to method triggerRendezvous of the test class, BytemanNGTests. This method has an empty body so normally calling it would do nothing

private void triggerRendezvous(PipelineProcessor processor)

{

// nothing to do here as Byteman will inject the

// relevant code into this method

}

However, when the Byteman rules are loaded a call to rendezvous will occur, passing $1, the pipeline element supplied in the call to triggerRendezvous, as argument to the call to rendezvous. The test method can choose to rendezvous with either pipeline processor simply by caling triggerRendezvous.

So. here is the rest of the implementation of test method testForwardPublishingGood

. . .

triggerRendezvous(binder);

String value = bindings.get("X4");

assert("window".equals(value));

triggerRendezvous(replacer);

cssource.join();

binder.join();

replacer.join();

cssink.join();

String output = cssink.toString();

assert(output.equals("the boy threw the stick for the dog to catch\n" +

"a boy broke a window with a stick\n" +

"the boy threw the stick at the window\n"));

}

When the test thread returns from the first call to triggerRendezvous it can be sure that the Binder is about to start processing the 3rd line. When it returns from the second call to triggerRendezvous it can be sure that the 3rd line has been processed and hence that binding X4 has been added to the bindings map. Hence the assertion which follows these two calls should not fail.

The test thread can also be sure that the BindingReplacer has not processed the second line at this point because that cannot happen until the test thread exits from the 3rd call to triggerRendezvous. The BindingReplacer will be forced to wait at this rendezvous until the test thread arrives. The final call to triggerRendezvous allows the BindingReplacer to exit the rendezvous after processing the 3rd line. Since the binding for X4 was present when it executed this line the ouptut should include a substitution for this variable as asserted at the end of the method.

The second test method, testForwardPublishingBad, is a minor variation on this first one. It shoulds be obvious how it manages to reverse the execution order.

@BMScript(value="timing", dir="target/test-classes")

@Test()

public void testForwardPublishingBad() throws Exception

{

System.out.println("testForwardPublishingBad:");

BindingMap bindings = new BindingMap();

StringBuffer buffer = new StringBuffer("the boy threw the stick for the dog to catch\n");

buffer.append("a ${X1} broke a ${X4} with a ${X2}\n");

buffer.append("the boy threw the stick at the window\n");

CharSequenceSource cssource = new CharSequenceSource(buffer);

Binder binder = new Binder("the ([A-Za-z]+)", "X", bindings, reader);

BindingReplacer replacer = new BindingReplacer(bindings, binder);

CharSequenceSink cssink = new CharSequenceSink(replacer);

cssource.start();

binder.start();

replacer.start();

cssink.start();

triggerRendezvous(replacer);

String value = bindings.get("X4");

assert(value == null);

triggerRendezvous(binder);

cssource.join();

binder.join();

replacer.join();

cssink.join();

String output = cssink.toString();

assert(output.equals("the boy threw the stick for the dog to catch\n" +

"a boy broke a ${X4} with a stick\n" +

"the boy threw the stick at the window\n"));

}

This time the first pair of calls to triggerRendezvous allow the BindingReplacer to transform the second line before the binding for X4 has been inserted. The Binder is not able to process the 3rd line and insert this binding until the 3rd call to triggerRendezvous has been competed. This is reflected in the final output where the reference to variable X4 remains unsubstituted as asserted.

To run the tests using maven you execute the following command in the top level directory

> mvn test -P testng

. . .

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running TestSuite

byteman jar is /ssd/home/adinn/.m2/repository/org/jboss/byteman/byteman/2.2.1/byteman-2.2.1.jar

Setting org.jboss.byteman.allow.config.update=true

testForwardPublishingGood:

#INPUT

the boy threw the stick for the dog to catch

a ${X1} broke a ${X4} with a ${X2}

the boy threw the stick at the window

#END

install(X1 -> boy)

bind(X1 -> boy)

install(X2 -> stick)

bind(X2 -> stick)

install(X3 -> dog)

bind(X3 -> dog)

reject(X4 -> boy)

reject(X4 -> stick)

install(X4 -> window)

bind(X4 -> window)

bindingReplace(a ${X1} broke a ${X4} with a ${X2} -> a boy broke a window with a stick)

#OUTPUT

the boy threw the stick for the dog to catch

a boy broke a window with a stick

the boy threw the stick at the window

#END

testForwardPublishingBad:

#INPUT

the boy threw the stick for the dog to catch

a ${X1} broke a ${X4} with a ${X2}

the boy threw the stick at the window

#END

install(X1 -> boy)

bind(X1 -> boy)

install(X2 -> stick)

bind(X2 -> stick)

install(X3 -> dog)

bind(X3 -> dog)

bindingReplace(a ${X1} broke a ${X4} with a ${X2} -> a boy broke a ${X4} with a stick)

reject(X4 -> boy)

reject(X4 -> stick)

install(X4 -> window)

bind(X4 -> window)

#OUTPUT

the boy threw the stick for the dog to catch

a boy broke a ${X4} with a stick

the boy threw the stick at the window

#END

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.361 sec

. . .

Alternatively you can execute the following ant command

> ant testng

. . .

Where Can I Download The Tutorial Sources?

The tutorial sources are available from the JBoss downloads server as a zip file or from github

In order to run the example application main programs and tests provided as part of this downlaod you will need to install a JDK6 (or greater) release which supports the dynamic agent load capability. This includes OpenJDK, Oracle's Hotspot JDK and recent releases of the JRockit JDK (version jrmc-4.0.1- is known to work). You should ensure that the Java binaries in the JDK bin directory are available in your PATH.

You will also need to install either ant or maven, This tutorial has been tested using ant 7.1 and maven 3.0.2 but it may possibly work with earlier versions. Once again you shoudl ensure the ant or maven binaries are available in your path.

If you are building and running the test programs with maven you do not need to downlaod and install a Byteman release as maven will automatically pull the relevant jars from the network. However, you will need to configure maven to locate the jars on the public JBoss repository located at

http://repository.jboss.org/nexus/content/groups/public

If you are building and running the test programs with ant then you will need to download Byteman release 2.1.1 or greater. The latest release is available from the Byteman downloads page

http://www.jboss.org/byteman/downloads

Unzip the release and set environment variable BYTEMAN_HOME to identify the top-level directory of the resulting directory tree. It should contain several subdirectories including lib and bin.

JBossDeveloper

Fault Injection Testing With Byteman

You should already be familiar with Byteman and BMUnit . . .

What is Fault Injection Testing?

Can I See a Simple Example of Fault Injection?

An Example Pipeline Processor Application

A Very Testing Problem

The Pipeline Core Classes

PipelineProcessor Core SubClasses

Non-Core Library Classes

A Simple Test of the Pipeline Library Functionality

How Can We Trace Pipeline Activity?

Using BMUnit to Inject the Trace Rules

Breaking the Pipeline Application Using Byteman

Running the Fault Injection Code

Breaking and Entering a Timing Window

Where Can I Download The Tutorial Sources?

Comments