1 2 Previous Next 25 Replies Latest reply on Nov 14, 2006 3:05 PM by clebert.suconic

    Application Server Integration Tests

    ovidiu.feodorov

      This is the companion discussion thread for http://jira.jboss.org/jira/browse/JBMESSAGING-553. All issues related to the AS JMS integration tests and their relation to Messaging should be discussed here.

        • 1. Application Server Integration Tests
          rachmato

          I'd like to discuss the re-factoring of existing JBossMQ JMS tests in order to make them generic JMS tests - so that one test can be reun against different JMS providers.

          I had a look at several of the JBossMQ JMS test cases in the AS testsuite, and it seems that its possible to create a set of generic JMS test cases by doing the following:
          (i) use JMS provider neutral ObjectName domain names to name destinations within test cases (e.g. jboss.jms.destination instead of jboss.jbossmq.destination)
          (ii) provide a JMS provider netural Admin interface to be used to perform JMS provider administrative functions within test cases, and provide an AdminFactory to plug in JMS provider specific Admin impelemtations
          (iii) provide JMS provider neutral access to the AS and things like JNDI, JMX, deployment functions

          This is a combination of (i) the strategy of JORAM to use a JMS-provider neutal Admin interface to facilitate JMS provider administrative functions within test cases, and (ii) the strategy of subclassing from JBossTestCase to get access to the AS from within test cases.

          In such test cases, Topics and Queues would only be allocated via calls to the Admin interface, and not via deployments as is currently done in many test cases. Test cases could make use of AS functionality, such as deploying MDBs, etc. If other utility functions were required, these could be included via subclasses interposed between JBossTestCase and the actual test case. The correct admin class would be obtained via an AdminFactory and property, as in the JORAM scheme.

          However, I know that there has been previous work on providing a scheme for sharing JMS test cases: http://www.jboss.org/index.html?module=bb&op=viewtopic&t=72570, more ambitious than the simple scheme outlined above and with features I don't yet fully understand (but would like to find out about).

          Are there cases we are going to run into with JMS integration testing in which prohibit use of the basic scheme above? It's primary advantage is that its simple and easily maintainable.

          • 2. Re: Application Server Integration Tests
            rachmato

            I have had a look at the existing framework for incorporating Joram tests into the testsuite. As it is a candidate approach for the integration of generic JMS test cases into the testuite, i'd like to summarise how it works and what I feel are its advantages and disadvantages.

            The framework involves defining several sets of classes. The class hierarchy looks like this:

            --------------------------------------------------------------------------------------------------------------
            TestCase
            AbstractTestCase
            AbstractTestCaseWithSetup TestSetup
            * -----------------------> AbstractTestSetup
            * --------------> AbstractTestDelegate
            JoramTestDelegate Admin
            GenericAdmin AbstractAdmin
            JoramTestCase JBossASJMSAdmin
            JBossMQJoramUnitTestCase JBossMQAdmin
            ----------------------------------------------------------------------------------------------------------------

            Each column represents an inheritance hierarchy. The classes to the right based on Admin are a separate hierarchy. An asterisk and dotted arrow is intended to represent a member object, pointing to its type in the hierarchy (i.e. AbstractTestSetup holds an instance of JoramTestDelegate)

            How is it used? (llosely speaking)
            1. You implement a subclass of JoramTestCase (e.g. JBossMQJoramUnitTestCase) which implements the method getDelegate().
            This method specifies the name of the Admin implementation you want to use .
            2. You implement a JMS provider-specific class implemnting the Admin interface as a subclass of JBossASJMSAdmin (e.g. JBossMQAdmin)
            3. You write your generic JMS tests using JBossMQJoramUnitTestCase as your baseclass, and include code snippet similar to

            Admin admin = AdminFactory.getAdmin() ;
            admin.createQueueConnection(TEST_QUEUE) ;

            to create JMS destinations and connections.

            How does it work?
            1. The Joram test cases are hardwired to use an AdminFactory to load the class GenericAdmin as the Admin implementation.
            This class holds a member object which is initialised at run time and to which all JMS admin functions are delegated.
            2. Whether the JMS tests to be executed are individual tests or a test suite, the AbstractTestSetup class gets called
            and is reponsible for ensuring that the correct implementation of JBossASAdmin gets initialised in GenericAdmin.
            3. The JMS test cases don't need to do anything special other than use the code snippet above.

            This approach is used and works well for incorporating the Joram tests into the testsuite.

            Problems with this approach:
            1. JoramTestCase does not subclass from JBossTestCase, and so all calls to methods defined in JBossTestCase already present in
            the existing test cases will have to be recoded. JBossASJMSAdmin does incorporate an instance of JBossTestServices within
            it, but it does not provide an interface to all of the functions available in JBossTestCase.

            2. In order for AbstractTestSetup to be called when executing a test suite, the test suite needs to be wrapped in an
            instance of AbstractTestCase. Forgetting to do this will result in the Admin class not being correctly initialised.

            3. It takes time to find out how the class hierarchy works (for example, processing of test cases follows a different
            procesing path than for test suites), and this makes debugging and maintaining more complex.

            All in all, I am in favour of using a simpler approach, based on the simplified scheme described above, where test cases
            continue to sublass from JBossTestCase, and the only changes to test cases required are (i) the loading of the Admin class
            based on AdminFactory and specifying the Admin implementation via the property jms.provider.admin.class, (ii) replacing
            JMS specific calls to create destinations with calls to Admin (iii) making all desintaion names JMS provider neutral.

            This seems to me to be a faster approach and one which will work. If I have missed some obvious advantage of the more
            complex scheme, or am misrepresenting it, please point it out. I plan to begin converting one or two JBossMQ tests to the generic form based on
            the simple scheme over the next few days to see that everything works in practice.

            • 3. Re: Application Server Integration Tests
              ovidiu.feodorov

               

              rachmatowicz@jboss.com wrote:

              I had a look at several of the JBossMQ JMS test cases in the AS testsuite, and it seems that its possible to create a set of generic JMS test cases by doing the following:
              (i) use JMS provider neutral ObjectName domain names to name destinations within test cases (e.g. jboss.jms.destination instead of jboss.jbossmq.destination)


              Why do you need to use ObjectNames from within test cases? The test should be completely unaware that JBossMQ deploys its destinations as MBeans in the "jboss.mq.destination:" domain and Messaging uses "jboss.messaging.destination:" for the same purpose. The provider may not even deploy the destination as an MBean, if it chooses so. "jboss.jms.destination" won't work.



              • 4. Re: Application Server Integration Tests

                 

                "rachmatowicz@jboss.com" wrote:

                All in all, I am in favour of using a simpler approach, based on the simplified scheme described above, where test cases
                continue to sublass from JBossTestCase


                No the point is to write unit tests. You are not testing JBoss integration!

                The tests should be written against the microcontainer:
                http://www.jboss.com/index.html?module=bb&op=viewtopic&t=76657

                I said this should be based on a JMSAdmin like the JORAM tests.
                Not that it should use the JORAM admin implementation.

                Once we have our own admin implementation, the JORAM tests
                can use that as a delegate.

                The general outline should be something like:

                // A base class
                public abstract class AbstractJMSTest extends MicrocontainerTest
                {
                 /**
                 * Get the delegate
                 *
                 * @return the delegate
                 */
                 protected JMSTestDelegate getJMSDelegate()
                 {
                 return (JMSTestDelegate) getDelegate();
                 }
                
                 // helper methods that delegate/admin
                }
                


                // The delegate that encapsulates the admin/implementation
                public class JMSTestDelegate extends MicrocontainerTestDelegate
                {
                 static JMSAdmin admin;
                
                 static
                 {
                 // Possible rules
                 // 1) Look at system property
                 // 2) Is JBoss Messaging in the classpath
                 // 3) Is JBossMQ in the classpath
                 admin = initialiseAdmin();
                 }
                }
                


                // Real tests just use the infrastructure
                public class RealTest extends AbstractJMSTest
                {
                 public void testSomething()
                 {
                 ConnectionFactory cf = super.getConnectionFactory();
                 Queue queue = super.createQueue("something");
                 ...
                 }
                }
                



                • 5. Re: Application Server Integration Tests

                  I'd also recommend that most tests bootstrap
                  a default configuration for the server.

                  The default Microcontainer delegate looks for an xml file
                  based on the test class name.
                  e.g.
                  org.jboss.test.jms.test.TestSomething.class
                  uses
                  org.jboss.test.jms.test.TestSomething.xml

                  Rather than writing one for each test, there should be
                  a mechanism to provide a default JMS config
                  (while still allowing individual tests to use different configurations).

                  e.g. maybe?

                  public class JMSTestDelegate extends MicrocontainerTestDelegate
                  {
                   static JMSAdmin admin;
                  
                   ...
                  
                   protected void setUp() throws Execption
                   {
                   super.setUp();
                  
                   // No test specific configuration?
                   if (getBean("ConnectionFactory") == null)
                   {
                   String name = admin.getDefaultConfigName();
                   URL url = clazz.getResource(name); // the config will be in the classpath
                   deploy(url);
                   }
                   }
                  }
                  


                  Or perhaps:
                   protected void setUp() throws Execption
                   {
                   super.setUp();
                  
                  
                   String name = admin.getConfig(clazz.getName());
                   // No test specific configuration?
                   if (name == null)
                   name = admin.getDefaultConfigName();
                   URL url = clazz.getResource(name); // the config will be in the classpath
                   deploy(url);
                   }
                  


                  • 6. Re: Application Server Integration Tests
                    rachmato

                     


                    The test should be completely unaware that JBossMQ deploys its destinations as MBeans in the "jboss.mq.destination:" domain and Messaging uses "jboss.messaging.destination:" for the same purpose. The provider may not even deploy the destination as an MBean, if it chooses so.


                    Yes, I agree.

                    • 7. Re: Application Server Integration Tests
                      rachmato

                       


                      No the point is to write unit tests. You are not testing JBoss integration!

                      The tests should be written against the microcontainer:
                      http://www.jboss.com/index.html?module=bb&op=viewtopic&t=76657


                      Could you please clarify what you mean by "No, the point is to write unit tests. You are not testing JBoss integration"?

                      All i'm trying to do is execute the existing JBossMQ integration tests against JBoss Messaging, in the most straightforward way. For me, that means in the way I outlined above. Is there a good reason why the Messaging destinations should not be deployed as MBeans, using JMX, as they currently are for JBossMQ?




                      • 8. Re: Application Server Integration Tests

                        The original task was to write a set of unit tests that can be reused.

                        Whether that is:
                        1) JBossMQ inside JBoss
                        2) JBoss Messaging inside JBoss
                        3) JBossMQ standalone
                        4) JBoss Messaging standalone
                        5) JBossMQ in EJB3 embeded
                        6) JBoss Messaging in EJB3 embedded
                        7) JBossMQ bootstrapped inside junit
                        8) JBoss Messaging bootstrapped inside junit
                        etc.

                        Which case applies depends upon which Admin you choose.

                        Using JBossTestCase and MBeans limits you to testing (1) and (2)
                        and there is a plan to move away MBean deployments anyway
                        since this doesn't work if you want to bootstrap JBoss Messaging
                        inside Tomcat or Weblogic, etc.

                        In short write to the Admin class as the integration point, not
                        JBossTestCase or -service.xml

                        • 9. Re: Application Server Integration Tests

                          I suggested to Ovidiu over a year ago that we should write
                          an admin layer based on JMS which would have greatly
                          simplified this task.

                          And would have also provided a unified management layer
                          for both JBossMQ and JBoss Messaging.

                          psuedo code:

                          Queue managementQueue = jndi.lookup("jmsadmin/AdminQueue");
                          QueueRequestor r = new QueueRequestor(managementQueue);
                          Message m = session.createMapMessage();
                          m.setString("operation", "createQueue");
                          m.setString("queueName", "testQueue");
                          m.setInteger("queueDepth", 1000);
                          Message response = r.request(m);
                          
                          Message m = session.createMapMessage();
                          m.setString("operation", "addUser");
                          m.setString("user", "ovidiu");
                          m.setString("password", "ovidiu");
                          m.setString("roles", "guest,subscriber,admin");
                          Message response = r.request(m);
                          
                          etc.
                          


                          • 10. Re: Application Server Integration Tests
                            rachmato

                            OK. I think i'm starting to get the picture of why the approach I outlined earlier isn't what you want - you want an approach which can be used in a variety of configurations in which the Messaging product can exist, and which doesn't rely on JMX being available.

                            What i'll do over the next day or two is to read up on the microkernel and how it works, then ask any questions I have, and then start to outline my understanding of your design. At that stage, i'll be able to begin the real work.

                            • 11. Re: Application Server Integration Tests
                              rachmato

                              I hope to present a detailed picture of my understanding of what you want tomorrow (Friday) which we can use to move forward.

                              One question though: from what I have read, the microcontainer (MC) has limitations, in particular that remote communication is not supported at present. I assume this means that, say, in a test case based on MicrocontainerTestCase, that the JMS provider, deployed destinations, and test case itself all have to reside in the same VM. (Please correct me if i'm mistaken).

                              So running test cases using the MC, despite this limitation, makes sense when we are outside the AS, as there is no alternative.

                              However, what if JMX (which supports remote communication) is available?. You mentioned in an earlier posting that tests should be written against the MC. Does this mean that you want to run these generic JMS tests against the MC, even when the AS is available? That is, in the AS test suite, to run all JMS tests (whether against the JBossMQ provider or Messaging preovider) by deploying these providers to the MC and not JMX?


                              • 12. Re: Application Server Integration Tests
                                ovidiu.feodorov

                                Adrian, integration with MC and all these grandiose plans are all fine, but don't you think it would me more practical to start with an initial first small step: separation of "generic" JMS tests from JBossMQ-specific functional tests, and making sure that the integration tests work with both MQ and Messaging using the simplest possbile refactoring that Richard suggested?

                                The net gain is that we have Messaging qualified for JBossAS ASAP, and nothing would preclude Richard (or somebody else) to build later in top of that, so eventually we reach the nirvana of integration tests.

                                • 13. Re: Application Server Integration Tests

                                   

                                  "ovidiu.feodorov@jboss.com" wrote:
                                  Adrian, integration with MC and all these grandiose plans are all fine, but don't you think it would me more practical to start with an initial first small step: separation of "generic" JMS tests from JBossMQ-specific functional tests, and making sure that the integration tests work with both MQ and Messaging using the simplest possbile refactoring that Richard suggested?


                                  No, for all reasons I have stated before and can't be bothered to
                                  repeat, again...

                                  • 14. Re: Application Server Integration Tests
                                    rachmato

                                    I looked at the recommendadtions for running tests against the MC (as opposed to JMX) and I feel that pursuing it is the right approach, but perhaps not right at this moment.

                                    Given that the MC will replace JMX, running test cases against the MC will have to happen, and not just for JMS tests. Any framework designed should also be applicable to other classes of test cases. At present, even with AB's detailed design sketches, it's not clear ( to me :-( ) how elements common to most test cases should be organised - whether by delegation or through the use of QueueRequestors. I think we should have a detailed design of this new framework, which will essentially supercede JBossTestCase, and agreement on the design, before embarking on such significant changes.

                                    Therefore, i'm going to start applying the simplified refactoring today, taking JMS test cases from the jbossmq directory, refactoring them, and placing the refactored versions in jbossmessaging directory. When this is complete, we will have a set of generic JMS tests which will apply to either JMS provider, running in the context of the AS.


                                    1 2 Previous Next