7 Replies Latest reply on Jan 25, 2006 5:19 PM by Ovidiu Feodorov

    Smoke test

    Ovidiu Feodorov Master

      I added a very rudimentary smoke test that will (hopefully) stop us from releasing sars that don't even deploy. Currently it creates a JBoss configuration from scratch, deploys the scoped sar and sends a simple JMX invocation into the ServerPeer using twiddle.

      To apply the smoke test:

      * Set JBOSS_HOME
      * Have the scoped sar ready in jboss-head/jms/output/lib (cd jboss-head/jms; ant clean; ant sar-scoped)

      and then

      cd jboss-head/jms/tests/bin
      ./smoke-test
      


      The build must succeed for the test to pass. smoke-test clean shuts down the instance and cleans the configuration.

      The process can be improved by modifying the script to:
      - deploy a destination
      - run a client that sends a message
      - run a client that receives the message
      - deploy an MDB,
      etc.

      Ideally, the clients should be our examples from docs, so they should be written in such a manner that they'll allow automatic testing.


        • 1. Re: Smoke test
          Ovidiu Feodorov Master

          Note: This is NOT supposed to be (or replace) an integration test.

          • 2. Re: Smoke test
            Tim Fox Master

            Previously I just ran the jms tests against a remote jboss instance to do the "smoke" test.

            Now the tests no longer allow you to do this, after the recent changes to the testsuite, so I agree we are in need of something like this.

            So far I have just been using simple custom test that connects to a remote jboss instance and does some simple jms operations.

            • 3. Re: Smoke test
              Ovidiu Feodorov Master

              The unit tests serve a different purpose. They are supposed to be automated so they can be ran by cruisecontrol. Which has a hard time running them in local mode, let alone starting a separate JBoss instance :)

              If you want to run them to validate the deployment sar with an external JBoss instance, it's very easy to re-enable this, I will just need to use a different ant property instead of "remote" to notify ServerManagement to use the JNP initial context. "external", or something like that.

              However, the need of smoke test like the one we have now (which it's not entirely complete, as I mentioned below) still remains, because our unit tests don't test complex interactions like deploying and MDB, for example.

              This is where the integration tests come into place, and the responsibilities of the smoke test will be replaced by those. Even then, we'll needed to validate a release, as a redundancy, if nothing else.

              • 4. Re: Smoke test
                Scott Stark Master

                An integration test we need is to to be able to run the existing jbossas unit tests against the new stack. We need to factor out the few jbossmq specific tests, but the bulk are vanilla jms tests. Currently such a configuration would require that a server config be created under testsuite/src/resources/test-configs, and the associated run starts/stops this configuration.

                Ideally such server configurations come from specifying the services to include, but we don't have such a profile notion available yet.

                • 6. Re: Smoke test
                  Ovidiu Feodorov Master

                  I completed the smoke test so now it reasonably tests a scoped sar with JBoss 4.x

                  During a full run, the test:

                  - cleans the working area
                  - builds a scoped sar from scratch
                  - creates a Messaging configuration under $JBOSS_HOME/server
                  - deploys and pings the scoped sar
                  - deploys and pings a scoped queue
                  - deploys and pings a scoped topic
                  - runs all examples from docs/examples (sending and receiving message from a queue, sending and received messages from a topic, deploying an unscoped MDB and sending messages to it)

                  To run it, just set JBOSS_HOME to point to the JBoss 4.x installation you want to use as environment and:

                  
                  cd jboss-head/jms/tests/smoke
                  ant
                  


                  If just one of these steps fails, the whole smoke tests fails.




                  No sar should be released (even after temporary fixes for on-going cases) without running a successful smoke test on it.





                  • 7. Re: Smoke test
                    Ovidiu Feodorov Master

                    I improved the smoke test script to be able to run the same tests on a jboss 5 instance. In order to execute a fully automated smoke test run on both JBoss 4 and JBoss 5, you only need to set the environment variables JBOSS4_HOME and JBOSS5_HOME (the script will complain and exit if they're not set).

                    Successful completion on the test run on JBoss 5 is contingent upon the availability of the correct JBoss Remoting version in the server's default configuration, so now the JBoss 5 smoke test will fail.