distributed test framework

[tom.elrod]

The distributed test framework under the remoting module tests can be used to start multiple client and server instances, along with a driver instance that will use a remote barrier for starting and stopping all the instances. This allows users to create a server and multiple client instances who will execute their client test code at the same time. The driver collects the junit assert results from the clients (and/or server).

This was created for remoting code testing because I need the following:

1. Ability to create one more clients that execute against a server at the same time.
2. A way to consolidate test failure reporting from clients and server.
3. Simulation of seperate processes between client and server for testing of remote classloading.
4. A single class that can be executed as a JUnit TestCase so that can be included within general testsuite run and failures reported within general testsuite results.

Am certainly not tied to dtf, but would need something that meets these requirements if go with something else.

The biggest problem with dtf currently is timing issues. Since the driver actually creates new instances for the clients and server to run in, and ultimately have not direct control of these after started, have to setup timeouts for the cases in which can not detect proper start up or shutdown of these new processes. These timeouts vary widely depending on initialization on startup and even on the hardware the tests run on (since can start up much slower on slower hardware).

[rajdeep_dua]

In JMS the requirements are

1.Synchrnize the send receive in different JVMs
2.Message sent by the Producer has to be compared to the one consumed by the consumer

The current DTF, provides mechanism only for remoteAssertions,our reuirements are for passing objects back and forth between different producers/consumers.Assertions can be done by the controller(which spawns the producer and consumer)

[ovidiu.feodorov]

It seems that JBoss Benchmarking (Clebert's work) is what we need. I would check it out from CVS and look at it. Is there a functional spec document around?

[ryan.campbell]

I don't think we have a functional spec, but the roadmap is in the JIRA Benchmarking project: http://jira.jboss.com/jira/browse/JBBENCH?report=com.atlassian.jira.plugin.system.project:roadmap-panel

At the bottom - MicroBenchmark Framework 1.0

[clebert.suconic]

I was looking to dtf when I was creating the framework. Actually the
config for JGroups I'm using was literally copied from Tom's work. :-).
And I'll be copying/using anything cool he has created. :-)


But the only problem is that the framework also needs lots of
improvements, and I was not being able to invest time as I'm focusing
in Specj and HP right now. (And we have now Intel coming to Specj also).

I have validated the idea with some people and everybody thinks the
idea for the framework is valid.

I've used decorators for submitting load, so you won't need to code
loops and Thread synchronizations. All you would have to do is to
decorate your testcase with a decorator and you have the loader done.
I'm considering a decorator for synchronizing multiple clients as well.

I have the Roadmap for the framework at:

http://jira.jboss.com/jira/secure/IssueNavigator.jspa?reset=true&pid=12310062&fixfor=12310214

What about you guys contribute to the framework? As you guys might
need to develop something, maybe adding effort in something generic is better.

The code is committed as jboss-benchmark in our public cvs.


Clebert