Stopping a JUnit test with Byteman

[rachmato]

Hello Byteman Users

I have a JUnit test involving a client and a server in which I want to detect a condition on the client and then stop the test based on that condition. In this particular case, I want to determine if a particular message handler was called on within some of the client code being tested and, if so, fail the test. The fact that the message handler was called indicates to me that the test was not successful.

I set up a Byteman script (together with BMUnit annotation) which detects that the message handler was called on the client and I can print out a trace message to the log when the condition is detected:

RULE check for message handler
CLASS  org.blah.MyMessageHandler
METHOD processMessage
AT EXIT
IF true
  DO debug("message handler called for invocation");
  # now stop the test
ENDRULE

I understand that any action performed by the script will be performed at that point in the execution, in this case, at the exit of the method.

I could throw an exception, but this message handler runs in a thread which is not the main thread of the test and so cannot be caught and used to stop the test.

Is there a standard solution for stopping a JUnit test when a condition is detected somewhere in the client code? Does another script need to be set up to wait for the condition to be detected?

Advice appreciated.

Richard 

[rachmato]

I decided to use a global flag variable (visible to all rules) to indicate when the message handler has been called. This flag can be set from any thread. I then added another rule, executed on the main test thread, which checks that flag at an appropriate time and raises an exception. The exception can then be handled to call Assert.fail() and fail the JUnit test.

[adinn]

Hi Richard,

Yes, you have discovered a very useful trick:  injecting rules into i) methods which are called by the app code under test and ii) a method implemented by the test thread provides a way of establishing a sideband communication or synchronization channel between test and app code.

A good example of how this can be used for synchronization is provided in the advanced Byteman fault injection tutorial. It injects rules which call rendezous at carefully selected points in the app code and also into a rendezvous method called by the test. The latter accepts a String identifying which rendezvous to meet at. Other rules are provided which allow a rendezvous to be enabled  and disabled. That allows the test thread to stop and start application threads at defined points during their execution, ensuring they proceed through timing windows in known orders.

Of course, the existing Byteman primitives are quite limited  as regards the sort of information channels they can open up. At present they can only really be used to communicate flag settings or counter values. However, you can easily add your own custom helpers to allow arbitrary objects to be transferred between test and app code. Indeed, I have thought about the idea of adding an arbitrary label method to the helper i.e.

  void label(Object label, Object target)

  Object labelTarget(Object label)

to enable arbitrary objects to be saved for later retrieval. This can be implemented simply by adding  <label, target> to a static hash table in the Helper class. label would allow you to use either a global label (e.g. "foo") or an application instance (e.g. a FileStream) to tag an object encountered during execution (a null target can be used to clear the hash table entry). The test thread can use labelTarget() to retrieve a labelled instance and ensure that it matches up to expectations. A list version might also be useful to build up collections:

  void labelList(Object label, Object target)

  void clearLabelList(Object label)

  List<Object> labelListTarget(Object label)

Alternatively,  you could use a combination of a counter and label to list multiple values

  label(o.toString() + incrementCounter(o), target)