9 Replies Latest reply on Apr 29, 2005 12:59 PM by Clebert Suconic

    Version compatibility tests

    Clebert Suconic Master

      As for http://jira.jboss.com/jira/browse/JBAS-1598, I'm creating a TestSuite that will execute a bunch of defined tests against 4.0.1 and 4.0.0.

      In other words, we will execute tests against current version using client libraries from 4.0.0 and them 4.0.1.

      The problem I'm having now is to define what's the basic set of tests I could use in that suite. Right now I'm using everything defined into "tests-standard-unit" and this seems to be huge.

      So... if you have any clue for shrinking the number of tests used int that list will be more than welcome. I guess we want basically test client libraries, so I guess we will have to execute mostly remote tests.

      Thanks,


      Clebert Suconic

        • 1. Re: Version compatibility tests
          Scott Stark Master

          Yes, that is what I wanted to test. Any client facing api needs to be tested, so remote ejbs, jms, webservices, jndi, jmx. These all have well defined packages in the testsuite.

          • 2. Re: Version compatibility tests
            Clebert Suconic Master

            My point is to find a minimal set of tests.

            For example in webservices,
            we have about 28 or 27 packages in the testsuite. (I've copied in this message for convenience).
            Do we need to exercize all of them to make sure the client API works?

            List of tests in web-services:

            org.jboss.test.webservice.admindevel
            org.jboss.test.webservice.attachment
            org.jboss.test.webservice.contextroot
            org.jboss.test.webservice.encstyle
            org.jboss.test.webservice.exception
            org.jboss.test.webservice.handlerflow
            org.jboss.test.webservice.header
            org.jboss.test.webservice.jbas897
            org.jboss.test.webservice.jbws124
            org.jboss.test.webservice.jbws128
            org.jboss.test.webservice.jbws153
            org.jboss.test.webservice.jbws64
            org.jboss.test.webservice.jbws68
            org.jboss.test.webservice.jbws70
            org.jboss.test.webservice.jbws71
            org.jboss.test.webservice.jbws79
            org.jboss.test.webservice.jbws82
            org.jboss.test.webservice.jbws83
            org.jboss.test.webservice.jbws84
            org.jboss.test.webservice.marshalltest
            org.jboss.test.webservice.message
            org.jboss.test.webservice.samples
            org.jboss.test.webservice.samples2
            org.jboss.test.webservice.samples2docclient
            org.jboss.test.webservice.version
            org.jboss.test.webservice.ws4eesimple
            org.jboss.test.webservice.wsdlimport

            • 3. Re: Version compatibility tests
              Scott Stark Master

              I would say all should be included. You can check with Thomas to see if some of these tests are excersizing features that really should not be considered as supportable across versions.

              • 4. Re: Version compatibility tests
                Clebert Suconic Master

                This is almost done, I'm just validating it before I commit the code.

                I have only one consideration. Some of the tests are requiring jboss.jar.

                As jboss.jar has all the classes contained into jboss-client.jar I'm not sure if I could add jboss.jar in the classpath for this tests as well.

                I don't know, it looks like some of these classes contained into jboss.jar should be present in jboss-client.jar as well, what would represent some point to be fixed.

                For example, org.jboss.test.client.test.MetaDataUnitTestCase requires org/jboss/metadata/ClientMetaData which is not represent into jboss-client.jar

                Any thoughts Scott?


                Clebert

                • 5. Re: Version compatibility tests
                  Scott Stark Master

                  A test that requires jboss.jar either is not an external client test (and MetaDataUnitTestCase is not an external client test that should be run in this compatibility testsuite), or it indicates a problem with the definition of the client jars.

                  Every unit test in this compatibility test run should be able to run using only the jars in the client directory of the dist.

                  • 6. Re: Version compatibility tests
                    Clebert Suconic Master

                    Ok!

                    Just as a FYI (Let me know if you foresee any problem):

                    I'm composing the classpath with:
                    - build.classes = compiled classes from the testsuite
                    - all the client libraries for a specific version
                    - library.classpath - as some of the classes are make usage of these libraries:
                    apache.avalon.classpath
                    apache.commons.classpath
                    apache.log4j.classpath
                    apache.xerces.classpath
                    apache.jaxme.classpath
                    wutka.dtdparser.classpath
                    dom4j.dom4j.classpath
                    oswego.concurrent.classpath
                    ibm.wsdl4j.classpath
                    jacorb.jacorb.classpath
                    jgroups.jgroups.classpath
                    junit.junit.classpath
                    junitejb.junitejb.classpath
                    javassist.classpath
                    sun.jaf.classpath
                    sun.javamail.classpath
                    sun.servlet.classpath
                    trove.classpath
                    gnu.regexp.classpath
                    apache.bcel.classpath
                    hsqldb.hsqldb.classpath
                    hibernate2.classpath
                    odmg.classpath
                    cglib.classpath

                    • 7. Re: Version compatibility tests
                      Scott Stark Master

                      Yes, that's what we want to do. This also helps test another compatibility issue with classes missing from client jars that has shown up from time to time due to the promiscuous use of the jboss module classes as the testuite classpath.

                      • 8. Re: Version compatibility tests
                        Clebert Suconic Master

                        The code is already commited into Branch_4_0.

                        The matrix testsuite is part of the regular testsuite.

                        The only requirement for this is to have defined a variable -Dmatrix-versions.

                        I'm trying to fix cruise control testsuite for jboss-4.0 and configure the matrix execution against:


                        - jboss-4.0. (It's really broken anyway. I guess the only thing that passed was jbossMX. I will remove as soon as people are familiar with this)
                        - jboss-4.1-sp1
                        - we will always check the current version just to validate classpath issues