12 Replies Latest reply on Mar 18, 2006 7:49 PM by starksm64

    Messaging Integration Tests

    ovidiu.feodorov

       

      Scott Stark wrote:
      An integration test we need is to to be able to run the existing jbossas unit tests against the new stack. We need to factor out the few jbossmq specific tests, but the bulk are vanilla jms tests. Currently such a configuration would require that a server config be created under testsuite/src/resources/test-configs, and the associated run starts/stops this configuration.

      Ideally such server configurations come from specifying the services to include, but we don't have such a profile notion available yet.


        • 1. Re: Messaging Integration Tests
          ovidiu.feodorov

          I locally modified the JBoss head testsuite so now it runs a set of integration tests using Messaging instead of JBossMQ as default JMS provider. Before committing, I have a few issues want to clear with those who know the testsuite well.

          • 2. Re: Messaging Integration Tests
            ovidiu.feodorov

            1.

            The testsuite doesn't currently have the concept of pluggable JMS provider, so, for example testsuite/src/resources/messaging contains two destination deployment descriptors (test-destinations-full-service.xml and test-destinations-service.xml) that are JBossMQ specific. These files are copied in bulk into testuite/output/resources and used as it by tests. There are other similar examples.

            The way I am handling this now is to define a new "messaging-tests" target, that besides creating a "messaging" JBoss configuration, effectively overwrites the output/resources descriptors with Messaging-specific ones. This is a temporary (and poor) solution, since it breaks any further testsuite run, unless "clean" is performed first.

            A better way of implementing this, as I see it, is to define a "jms.provider" testsuite build level system property that can take arbitrary values ("messaging", "jbossmq", "momandpopjms", ...) and then modify all affected target to take this property into account when they do their stuff.

            What is your take on this?

            • 3. Re: Messaging Integration Tests
              ovidiu.feodorov

              2.

              This is the "JMS integration" test subset came up with. Are there any others?

               <patternset id="messaging.includes">
               <include name="org/jboss/test/client/test/*TestCase.class"/>
               <include name="org/jboss/test/cts/test/MDBUnitTestCase.class"/>
               <include name="org/jboss/test/foedeployer/test/*TestCase.class"/>
               <include name="org/jboss/test/jmsra/test/*TestCase.class"/>
               <include name="org/jboss/test/mdb/test/*TestCase.class"/>
               <include name="org/jboss/test/messagedriven/test/*TestCase.class"/>
               <include name="org/jboss/test/security/test/*TestCase.class"/>
               <include name="org/jboss/test/timer/test/*TestCase.class"/>
               </patternset>
              




              • 4. Re: Messaging Integration Tests
                ovidiu.feodorov

                3.

                Currently Messaging is using a jboss-remoting.jar snapshot because we fix bugs as we go, and the fixed Remoting jar doesn't make it into repository fast enough. Obviously, when running the testsuite, jboss.remoting.classpath contains the old archive so all messaging tests that involve remoting fail.

                What would be a generic mechanism to deal with this?

                • 5. Re: Messaging Integration Tests
                  ovidiu.feodorov

                  4.

                  I added a new "messaging-tests" target and I am invoking it as follows:

                   <target name="tests" description="Execute all non-benchmark tests." depends="maybejars">
                   <record name="${basedir}/build.log" append="no" action="start" loglevel="${buildlog.level}"/>
                   <property name="nojars" value="true"/>
                   <antcall target="jboss-minimal-tests" />
                   <antcall target="jboss-all-config-tests"/>
                  
                   .....
                  
                   <antcall target="tests-webservice-ssl"/>
                   <antcall target="messaging-tests"/>
                   <antcall target="tests-report"/>
                   <record name="${basedir}/build.log" action="stop"/>
                   </target>
                  
                  


                  Won't "messaging-tests" results overwrite previous test run results, for example "jboss-all-config-tests"? How can I tell by just looking at the final report which tests was run as part of a messaging run and which test was run as part of "jboss-all-config-tests" run?





                  • 6. Re: Messaging Integration Tests
                    ovidiu.feodorov

                    5.

                    I see JBossMQ specific tests in the testsuite. Don't they belong under jboss-head/messaging/tests ?


                    • 7. Re: Messaging Integration Tests

                      1. The jms.provider is a better way to deal with this.
                      See the "jms admin" I did for integrating the joram tests.
                      This is the way to go longer term.

                      2.There are quite a few tests that use JMS, rather than test it

                      3. I told you this would bite you. :-)
                      You can't defer the fact that JMS can't decide thirdparty versions forever

                      4. See what has been done for the JACC tests

                      5. Is this the same question as 2?

                      • 8. Re: Messaging Integration Tests
                        starksm64

                         

                        "ovidiu.feodorov@jboss.com" wrote:
                        1.

                        The testsuite doesn't currently have the concept of pluggable JMS provider, so, for example testsuite/src/resources/messaging contains two destination deployment descriptors (test-destinations-full-service.xml and test-destinations-service.xml) that are JBossMQ specific. These files are copied in bulk into testuite/output/resources and used as it by tests. There are other similar examples.

                        The way I am handling this now is to define a new "messaging-tests" target, that besides creating a "messaging" JBoss configuration, effectively overwrites the output/resources descriptors with Messaging-specific ones. This is a temporary (and poor) solution, since it breaks any further testsuite run, unless "clean" is performed first.

                        A better way of implementing this, as I see it, is to define a "jms.provider" testsuite build level system property that can take arbitrary values ("messaging", "jbossmq", "momandpopjms", ...) and then modify all affected target to take this property into account when they do their stuff.

                        What is your take on this?


                        There should be some dynamic deployment of the destinations/configuration via the jsr88 or main deployer/rmi adaptor service with the deployment selected via provider id as suggested.


                        • 9. Re: Messaging Integration Tests
                          starksm64

                           

                          "ovidiu.feodorov@jboss.com" wrote:
                          2.

                          This is the "JMS integration" test subset came up with. Are there any others?



                          All of the org.jboss.test.jbossmq.* that don't in fact depend on jbossmq apis (and this is the majority of the tests) should be included.


                          • 10. Re: Messaging Integration Tests
                            starksm64

                             

                            "ovidiu.feodorov@jboss.com" wrote:
                            3.

                            Currently Messaging is using a jboss-remoting.jar snapshot because we fix bugs as we go, and the fixed Remoting jar doesn't make it into repository fast enough. Obviously, when running the testsuite, jboss.remoting.classpath contains the old archive so all messaging tests that involve remoting fail.

                            What would be a generic mechanism to deal with this?


                            In general the testsuite has to be running off the jars specified via jbossas thirdparty versions. The testsuite cannot be overriding these. What are you describing is an unstable release of remoting will result in failed integration tests until its updated. That is the way it is. If you want faster synch times then you need to push the changes back to remoting and get the snapshot updated faster.


                            • 11. Re: Messaging Integration Tests
                              starksm64

                               

                              "ovidiu.feodorov@jboss.com" wrote:
                              4.

                              I added a new "messaging-tests" target and I am invoking it as follows:
                               <target name="tests" description="Execute all non-benchmark tests." depends="maybejars">
                               <record name="${basedir}/build.log" append="no" action="start" loglevel="${buildlog.level}"/>
                               <property name="nojars" value="true"/>
                               <antcall target="jboss-minimal-tests" />
                               <antcall target="jboss-all-config-tests"/>
                              
                               .....
                              
                               <antcall target="tests-webservice-ssl"/>
                               <antcall target="messaging-tests"/>
                               <antcall target="tests-report"/>
                               <record name="${basedir}/build.log" action="stop"/>
                               </target>
                              
                              


                              Won't "messaging-tests" results overwrite previous test run results, for example "jboss-all-config-tests"? How can I tell by just looking at the final report which tests was run as part of a messaging run and which test was run as part of "jboss-all-config-tests" run?



                              This is not how to rerun sets of tests. See the run-junit task is used with a junit.configuration parameter defined to produce a unique test output:

                               <run-junit junit.patternset="deployment-service.includes"
                               junit.configuration="${jboss-junit-configuration}"
                               />
                              


                              Clebert created a org.jboss.ant.taskdefs.XMLJUnitMultipleResultFormatter that creates unqiue report file names based on a jboss-junit-configuration system property.


                              • 12. Re: Messaging Integration Tests
                                starksm64

                                 

                                "ovidiu.feodorov@jboss.com" wrote:
                                5.
                                I see JBossMQ specific tests in the testsuite. Don't they belong under jboss-head/messaging/tests ?


                                Yes. If these are pulled out all of the remaining org.jboss.test.jbossmq tests are just jms api tests.