Hello QA members

As Ryan has mentioned, i'm presently working as an intern, helping Mark Little migrate the Distributed Test Framework (DTF) to Atlanta. My understanding of the DTF and the details of its place in the QA process is not yet complete, so please bear with me.

The Distributed Test Framework (DTF) is a system for defining, executing and reporting on test cases and test suites, developed at Arjuna and used successfully as part of their QA process over the past three years. The system was intended for use in a distributed context, and has a distributed architecture: an instance of the system consists of a particular configuration of RMI-based services (Coordinator, TestNode, ServiceRegistry, ProductRepository, ReportCollector), typically residing on different machines, and these services cooperate in the execution of test cases. Broadly speaking, a set of test cases is presented using a web-based interface to the Coordinator, which schedules those test cases for execution on one or more TestNodes. Each TestNode represents a distinct hardware/OS/JVM/library configuration. The ServiceRegistry and ProductRepository are used by the Coordinator to discover the available TestNode configurations and product requirements, respectively, and schedule test cases accordingly. All test results are conveyed to the ReportCollator for the production of reports. The test engineer then views the test results using a web-based interface. This system permits the concurrent testing of multiple products over multiple hardware/OS/JVM/library configrations, in a way which exploits parallelism and access to the resources of a potentially large set of machines.

One significant difference between the JBoss AS testsuite and the DTF concerns the set of test tools used and the way in which test cases are defined. The JBoss AS testsuite is based on using the JUnit test framework for defining test cases and test suites, ant for executing those test cases/suites, and JUnitReport for the production of test reports from test case report logs. The DTF does not make use of these tools. In the DTF, "unit tests are defined by (i) writing one or more Java classes that execute to perform the test, and (ii) writing an xml test definition document which describes how the Java classes should be executed in order to perform the test". This test definition is then used by the Coordinator in scheduling the test. The above description lacks detail, but it is the best I can give at the moment. I'll know more once I get the DTF up and running. However, given a test case from the JBoss AS test suite, it appears as though significant rework would be required in order to run the test case under the DTF. One idea i'm toying with is to look for a way to adjust the definition of test cases in DTF so that they are JUnit-compatible, perhaps by defining an extension to JUnit which takes the distributed nature of the DTF into account.

Richard