-
1. Re: Messaging Integration Tests
ovidiu.feodorov Mar 18, 2006 3:58 PM (in response to ovidiu.feodorov)I locally modified the JBoss head testsuite so now it runs a set of integration tests using Messaging instead of JBossMQ as default JMS provider. Before committing, I have a few issues want to clear with those who know the testsuite well.
-
2. Re: Messaging Integration Tests
ovidiu.feodorov Mar 18, 2006 4:01 PM (in response to ovidiu.feodorov)1.
The testsuite doesn't currently have the concept of pluggable JMS provider, so, for example testsuite/src/resources/messaging contains two destination deployment descriptors (test-destinations-full-service.xml and test-destinations-service.xml) that are JBossMQ specific. These files are copied in bulk into testuite/output/resources and used as it by tests. There are other similar examples.
The way I am handling this now is to define a new "messaging-tests" target, that besides creating a "messaging" JBoss configuration, effectively overwrites the output/resources descriptors with Messaging-specific ones. This is a temporary (and poor) solution, since it breaks any further testsuite run, unless "clean" is performed first.
A better way of implementing this, as I see it, is to define a "jms.provider" testsuite build level system property that can take arbitrary values ("messaging", "jbossmq", "momandpopjms", ...) and then modify all affected target to take this property into account when they do their stuff.
What is your take on this? -
3. Re: Messaging Integration Tests
ovidiu.feodorov Mar 18, 2006 4:02 PM (in response to ovidiu.feodorov)2.
This is the "JMS integration" test subset came up with. Are there any others?<patternset id="messaging.includes"> <include name="org/jboss/test/client/test/*TestCase.class"/> <include name="org/jboss/test/cts/test/MDBUnitTestCase.class"/> <include name="org/jboss/test/foedeployer/test/*TestCase.class"/> <include name="org/jboss/test/jmsra/test/*TestCase.class"/> <include name="org/jboss/test/mdb/test/*TestCase.class"/> <include name="org/jboss/test/messagedriven/test/*TestCase.class"/> <include name="org/jboss/test/security/test/*TestCase.class"/> <include name="org/jboss/test/timer/test/*TestCase.class"/> </patternset>
-
4. Re: Messaging Integration Tests
ovidiu.feodorov Mar 18, 2006 4:04 PM (in response to ovidiu.feodorov)3.
Currently Messaging is using a jboss-remoting.jar snapshot because we fix bugs as we go, and the fixed Remoting jar doesn't make it into repository fast enough. Obviously, when running the testsuite, jboss.remoting.classpath contains the old archive so all messaging tests that involve remoting fail.
What would be a generic mechanism to deal with this? -
5. Re: Messaging Integration Tests
ovidiu.feodorov Mar 18, 2006 4:06 PM (in response to ovidiu.feodorov)4.
I added a new "messaging-tests" target and I am invoking it as follows:<target name="tests" description="Execute all non-benchmark tests." depends="maybejars"> <record name="${basedir}/build.log" append="no" action="start" loglevel="${buildlog.level}"/> <property name="nojars" value="true"/> <antcall target="jboss-minimal-tests" /> <antcall target="jboss-all-config-tests"/> ..... <antcall target="tests-webservice-ssl"/> <antcall target="messaging-tests"/> <antcall target="tests-report"/> <record name="${basedir}/build.log" action="stop"/> </target>
Won't "messaging-tests" results overwrite previous test run results, for example "jboss-all-config-tests"? How can I tell by just looking at the final report which tests was run as part of a messaging run and which test was run as part of "jboss-all-config-tests" run? -
6. Re: Messaging Integration Tests
ovidiu.feodorov Mar 18, 2006 4:07 PM (in response to ovidiu.feodorov)5.
I see JBossMQ specific tests in the testsuite. Don't they belong under jboss-head/messaging/tests ? -
7. Re: Messaging Integration Tests
adrian.brock Mar 18, 2006 6:50 PM (in response to ovidiu.feodorov)1. The jms.provider is a better way to deal with this.
See the "jms admin" I did for integrating the joram tests.
This is the way to go longer term.
2.There are quite a few tests that use JMS, rather than test it
3. I told you this would bite you. :-)
You can't defer the fact that JMS can't decide thirdparty versions forever
4. See what has been done for the JACC tests
5. Is this the same question as 2? -
8. Re: Messaging Integration Tests
starksm64 Mar 18, 2006 7:32 PM (in response to ovidiu.feodorov)"ovidiu.feodorov@jboss.com" wrote:
1.
The testsuite doesn't currently have the concept of pluggable JMS provider, so, for example testsuite/src/resources/messaging contains two destination deployment descriptors (test-destinations-full-service.xml and test-destinations-service.xml) that are JBossMQ specific. These files are copied in bulk into testuite/output/resources and used as it by tests. There are other similar examples.
The way I am handling this now is to define a new "messaging-tests" target, that besides creating a "messaging" JBoss configuration, effectively overwrites the output/resources descriptors with Messaging-specific ones. This is a temporary (and poor) solution, since it breaks any further testsuite run, unless "clean" is performed first.
A better way of implementing this, as I see it, is to define a "jms.provider" testsuite build level system property that can take arbitrary values ("messaging", "jbossmq", "momandpopjms", ...) and then modify all affected target to take this property into account when they do their stuff.
What is your take on this?
There should be some dynamic deployment of the destinations/configuration via the jsr88 or main deployer/rmi adaptor service with the deployment selected via provider id as suggested. -
9. Re: Messaging Integration Tests
starksm64 Mar 18, 2006 7:36 PM (in response to ovidiu.feodorov)"ovidiu.feodorov@jboss.com" wrote:
2.
This is the "JMS integration" test subset came up with. Are there any others?
All of the org.jboss.test.jbossmq.* that don't in fact depend on jbossmq apis (and this is the majority of the tests) should be included. -
10. Re: Messaging Integration Tests
starksm64 Mar 18, 2006 7:40 PM (in response to ovidiu.feodorov)"ovidiu.feodorov@jboss.com" wrote:
3.
Currently Messaging is using a jboss-remoting.jar snapshot because we fix bugs as we go, and the fixed Remoting jar doesn't make it into repository fast enough. Obviously, when running the testsuite, jboss.remoting.classpath contains the old archive so all messaging tests that involve remoting fail.
What would be a generic mechanism to deal with this?
In general the testsuite has to be running off the jars specified via jbossas thirdparty versions. The testsuite cannot be overriding these. What are you describing is an unstable release of remoting will result in failed integration tests until its updated. That is the way it is. If you want faster synch times then you need to push the changes back to remoting and get the snapshot updated faster. -
11. Re: Messaging Integration Tests
starksm64 Mar 18, 2006 7:47 PM (in response to ovidiu.feodorov)"ovidiu.feodorov@jboss.com" wrote:
4.
I added a new "messaging-tests" target and I am invoking it as follows:<target name="tests" description="Execute all non-benchmark tests." depends="maybejars"> <record name="${basedir}/build.log" append="no" action="start" loglevel="${buildlog.level}"/> <property name="nojars" value="true"/> <antcall target="jboss-minimal-tests" /> <antcall target="jboss-all-config-tests"/> ..... <antcall target="tests-webservice-ssl"/> <antcall target="messaging-tests"/> <antcall target="tests-report"/> <record name="${basedir}/build.log" action="stop"/> </target>
Won't "messaging-tests" results overwrite previous test run results, for example "jboss-all-config-tests"? How can I tell by just looking at the final report which tests was run as part of a messaging run and which test was run as part of "jboss-all-config-tests" run?
This is not how to rerun sets of tests. See the run-junit task is used with a junit.configuration parameter defined to produce a unique test output:<run-junit junit.patternset="deployment-service.includes" junit.configuration="${jboss-junit-configuration}" />
Clebert created a org.jboss.ant.taskdefs.XMLJUnitMultipleResultFormatter that creates unqiue report file names based on a jboss-junit-configuration system property. -
12. Re: Messaging Integration Tests
starksm64 Mar 18, 2006 7:49 PM (in response to ovidiu.feodorov)"ovidiu.feodorov@jboss.com" wrote:
5.
I see JBossMQ specific tests in the testsuite. Don't they belong under jboss-head/messaging/tests ?
Yes. If these are pulled out all of the remaining org.jboss.test.jbossmq tests are just jms api tests.