Yes, that is what I wanted to test. Any client facing api needs to be tested, so remote ejbs, jms, webservices, jndi, jmx. These all have well defined packages in the testsuite.
My point is to find a minimal set of tests.
For example in webservices,
we have about 28 or 27 packages in the testsuite. (I've copied in this message for convenience).
Do we need to exercize all of them to make sure the client API works?
List of tests in web-services:
I would say all should be included. You can check with Thomas to see if some of these tests are excersizing features that really should not be considered as supportable across versions.
This is almost done, I'm just validating it before I commit the code.
I have only one consideration. Some of the tests are requiring jboss.jar.
As jboss.jar has all the classes contained into jboss-client.jar I'm not sure if I could add jboss.jar in the classpath for this tests as well.
I don't know, it looks like some of these classes contained into jboss.jar should be present in jboss-client.jar as well, what would represent some point to be fixed.
For example, org.jboss.test.client.test.MetaDataUnitTestCase requires org/jboss/metadata/ClientMetaData which is not represent into jboss-client.jar
Any thoughts Scott?
A test that requires jboss.jar either is not an external client test (and MetaDataUnitTestCase is not an external client test that should be run in this compatibility testsuite), or it indicates a problem with the definition of the client jars.
Every unit test in this compatibility test run should be able to run using only the jars in the client directory of the dist.
Just as a FYI (Let me know if you foresee any problem):
I'm composing the classpath with:
- build.classes = compiled classes from the testsuite
- all the client libraries for a specific version
- library.classpath - as some of the classes are make usage of these libraries:
Yes, that's what we want to do. This also helps test another compatibility issue with classes missing from client jars that has shown up from time to time due to the promiscuous use of the jboss module classes as the testuite classpath.
The code is already commited into Branch_4_0.
The matrix testsuite is part of the regular testsuite.
The only requirement for this is to have defined a variable -Dmatrix-versions.
I'm trying to fix cruise control testsuite for jboss-4.0 and configure the matrix execution against:
- jboss-4.0. (It's really broken anyway. I guess the only thing that passed was jbossMX. I will remove as soon as people are familiar with this)
- we will always check the current version just to validate classpath issues