1 2 Previous Next 26 Replies Latest reply on May 6, 2005 12:25 PM by Tom Elrod

    Can we start writing jms test clients to check compliance wi

    rajdeep dua Newbie

      Hi
      Is the development in a state where jms api client can be written.i thought of start contribution by writing jms 1.1 compliance tests.

      saw a JMSClient in org.jboss.messaging.jms.tools, but not sure where is it going to pick up the initial context properties from(there is p2p.properties in resources).any pointers will be helpful

      IS there anything else where i can start contributing before diving into the core development.

        • 1. Re: Can we start writing jms test clients to check complianc
          rajdeep dua Newbie

          i was able to run a simple jms api test client with this

          Properties props = new Properties();
           props.put("java.naming.factory.initial","org.jboss.messaging.util.StandaloneInitialContextFactory");
           ctx = new InitialContext(props);


          please let me know if my writing test cases for jms compliance is ok
          i am using test package --
          org.jboss.test.messaging.jmsapi


          • 2. Re: Can we start writing jms test clients to check complianc
            Ovidiu Feodorov Master

            Thank you for your involvement.

            Yes, we are at the development stage when a JMS API client can be written, or at least, we're very close to that. Please look at the JMS Facade thread http://jboss.org/index.html?module=bb&op=viewtopic&t=61556, you'll see where we are. However, that client would have a very limited functionality. A lot of JMS API calls just throw NotYetImplementedException, so I am sure that at this stage, the client API won't pass any compliance test.

            The client you used, even if it is functional, it is deprecated, so I would suggest not to start testing anything on it, since we'll drop that code anyway.

            Having someone who specializes compliance tests, or other kind of testing for that matter, is essential, so your help is more than welcome. I hope we'll have something that will start passing compliance tests in several weeks.

            Until then, I would like you to start thinking at a more conprehensive testing and benchmarking harness that will grow toghether with the project.

            This is what I have in mind:

            We'll have four kinds of tests:
            - unit tests: written by the developers, at the same time or before writing the code itself. "Old" XP principle. They are essential, since they'll catch regression bugs and we'll allow us to refactor more freely.
            - compliance tests: your stuff. Ideally, I don't want developers involved with them, they should be performed by someone who is not closely involved with the code (or not yet), just because his/her only concern is to see how solid is the implementation from the JMS point of view.
            - integration test: test how well our JMS provider integrates with the AS; mainly the QA team's job.
            - stress tests and benchmarks: I hope to have a benchmarking framework that will allow us to closely follow key metrics for our implementation. As a very quick example, I am thinking of something like: how does our implementation behave when one hundred producers concurrently send 10KB messages? What are the key metrics to follow? How do these metrics evolve from a release to another? Does our implementation scale?


            Are you interested in getting involved with that?
            How did you think of doing compliance tests? SpecJ?



            • 3. Re: Can we start writing jms test clients to check complianc
              Adrian Brock Master

              I'd recommend reworking what is already in the testsuite
              to work over both JBossMQ and JBoss Messaging:
              http://cvs.sourceforge.net/viewcvs.py/jboss/jbosstest/src/main/org/objectweb/jtests/
              However, that is only jms1.0.2b level tests.

              Some of the JBossMQ tests are also pretty generic where they don't use
              internal apis for things like creating/clearing message queues, etc.
              http://cvs.sourceforge.net/viewcvs.py/jboss/jbosstest/src/main/org/jboss/test/jbossmq/

              Since JBoss Inc. is a j2ee licensee we have access to the JMS 1.1. TCK,
              but only our employees can run it.

              It is still a good idea to have a thorough testsuite for the developers who
              are not JBoss Inc employees or don't work for another j2ee licensee.

              • 4. Re: Can we start writing jms test clients to check complianc
                rajdeep dua Newbie

                Adrian objectwebs test cases you are talking about - are they same as the joram test suite.
                i had intergrated it with pramat's jms server 3 years back.

                i think they have upgraded the test suite for 1.1 version also.

                we can integrated those test cases.

                My first test client failed yesterday

                connection.getClientID()

                ClientID is always the tricky part

                throws OperationNotsupportedException. will post the exact stact trace.

                For performance test there is a grinder framework available
                Sonic also has a test harness which is downloadable.
                It covers many permutations/combinations outlined above.I can start looking into that also.

                please let me know where should i start posting testcases?


                • 5. Re: Can we start writing jms test clients to check complianc
                  rajdeep dua Newbie

                  For check on compliance i am following the specification itself.
                  each api call has various senarios where things can bomb or atleast there is a risk!

                  • 6. Re: Can we start writing jms test clients to check complianc
                    Ovidiu Feodorov Master

                    Try to build your compliance tests starting from what Adrian suggested
                    http://cvs.sourceforge.net/viewcvs.py/jboss/jbosstest/src/main/org/objectweb/jtests/
                    and
                    http://cvs.sourceforge.net/viewcvs.py/jboss/jbosstest/src/main/org/jboss/test/jbossmq/

                    No need to get into JBoss-specific API testing just yet. We'll do that later. If you could come up with a comprehensive JMS 1.1 test suite, that would be great.

                    • 7. Re: Can we start writing jms test clients to check complianc
                      rajdeep dua Newbie

                      As a first step towards compliance with 1.1, i have integrated the latest testsuite from Joram which is for 1.1 compliance

                      This i tested with JBoss 4.0 release code since the latest jms server is still not ready.

                      There are total of 186 tests in this test suite, 2 failed and there were errors in 2.

                      I had to modify

                      org.objectweb.jtests.providers.admin.JBossMQAdmin


                      Added te following methods
                      public void createConnectionFactory(String name) {
                       createQueueConnectionFactory(name);
                       }
                      
                       public void deleteConnectionFactory(String name) {
                       deleteQueueConnectionFactory(name);
                       }
                      

                      Here is the test summary report
                      commands, ant tests-objectweb-jms followed by ant tests-report-text in the testsuite module(note : i replaced the org.objectweb.* with the newer version from their CVS)

                      JBoss daily test results

                      SUMMARY

                      Number of tests run: 186

                      --------------------------------------------

                      Successful tests: 180

                      Errors: 2

                      Failures: 4

                      --------------------------------------------

                      [time of test: 2005-03-21.08-16 GMT]
                      [java.version: 1.4.2]
                      [java.vendor: Sun Microsystems Inc.]
                      [java.vm.version: 1.4.2-b28]
                      [java.vm.name: Java HotSpot(TM) Client VM]
                      [java.vm.info: mixed mode]
                      [os.name: Windows XP]
                      [os.arch: x86]
                      [os.version: 5.1]

                      Useful resources:

                      - http://jboss.sourceforge.net/junit-results/32/2005-03-21.08-16 for
                      the junit report of this test.


                      NOTE: If there are any errors shown above - this mail is only highlighting
                      them - it is NOT indicating that they are being looked at by anyone.

                      It is assumed that whoever makes change(s) to jboss that
                      break the test will be fixing the test or jboss, as appropriate!

                      --------------------------------------------
                      DETAILS OF ERRORS

                      Suite: org.objectweb.jtests.jms.conform.message.properties.JMSXPropertyTest
                      Test: testSupportsJMSXDeliveryCount
                      Type: failure
                      Exception: junit.framework.AssertionFailedError
                      Message: JMSXDeliveryCount property is not supported
                      ---------------------------------
                      Suite: org.objectweb.jtests.jms.conform.message.properties.JMSXPropertyTest
                      Test: testJMSXDeliveryCount
                      Type: failure
                      Exception: junit.framework.AssertionFailedError
                      Message: Message property 'JMSXDeliveryCount' not set.
                      ---------------------------------
                      Suite: org.objectweb.jtests.jms.conform.selector.SelectorSyntaxTest
                      Test: testInvalidIdentifiersStart
                      Type: error
                      Exception: org.jboss.mq.selectors.TokenMgrError
                      Message: Lexical error at line 1, column 1. Encountered: "%" (37), after : ""
                      ---------------------------------
                      Suite: org.objectweb.jtests.jms.conform.selector.SelectorTest
                      Test: unknown
                      Type: error
                      Exception: junit.framework.AssertionFailedError
                      Message: Timeout occurred
                      ---------------------------------
                      Suite: org.objectweb.jtests.jms.conform.session.QueueSessionTest
                      Test: testRollbackRececeivedMessage
                      Type: failure
                      Exception: junit.framework.ComparisonFailure
                      Message: expected:<...RollbackRececeivedMessage> but was:<...JMSDeliveryModeInSelector:1>
                      ---------------------------------
                      Suite: org.objectweb.jtests.jms.conform.session.UnifiedSessionTest
                      Test: testCreateDurableConnectionConsumerOnQueueConnection
                      Type: failure
                      Exception: junit.framework.AssertionFailedError
                      Message: Should throw a javax.jms.IllegalStateException
                      ---------------------------------

                      Please let me know how this can be checked in.

                      I think we might have to modify JBossMQAdmin if the MBean APIs etc are changing for the serverless impl.

                      Guys let me know your thoughts on this.


                      • 8. Re: Can we start writing jms test clients to check complianc
                        Adrian Brock Master

                         

                        "rajdeep_dua" wrote:

                        I think we might have to modify JBossMQAdmin if the MBean APIs etc are changing for the serverless impl.

                        Guys let me know your thoughts on this.


                        The idea would be to make the admin class configurable by a system property.
                        Then the tests could be run multiple times:
                        1) over JBossMQ
                        2) over serverless jms
                        3) over JBoss Messaging + JMS Facade
                        4) over JBoss + A.N.Other jms provider

                        • 9. Re: Can we start writing jms test clients to check complianc
                          Adrian Brock Master

                          rajdeep, if you send me your sourceforge id/email address we can
                          see about getting you cvs r/w to integrate this work into the testsuite.

                          • 10. Re: Can we start writing jms test clients to check complianc
                            rajdeep dua Newbie

                            Adrian,

                            My sourceforge id is: rajdeep_dua

                            My mail id is dua_rajdeep@yahoo.com

                            • 11. Re: Can we start writing jms test clients to check complianc
                              Ovidiu Feodorov Master

                              Rajdeep,

                              Please let me know when you start working on the test suite. I still don't recommend to apply it to the Mess (see http://www.jboss.org/index.html?module=bb&op=viewtopic&t=61556), but it's great if you get it ready to run on JBossMQ and you create the hooks for the Mess and other JMS provider.

                              • 12. Re: Can we start writing jms test clients to check complianc
                                rajdeep dua Newbie

                                Hi Ovidiu,
                                Started working on the test suite last week end.
                                (Had some issues with compiling the jmx module,hence the delay)

                                Here is the brief of the test suite structure
                                Location
                                ${jboss-head}/jms/tests
                                Package :
                                org.jboss.test.messaging.jmsapi
                                org.jboss.test.messaging.jmsapi.connection
                                org.jboss.test.messaging.jmsapi.flow
                                org.jboss.test.messaging.jmsapi.message
                                ...

                                Actors

                                All TestCases extend org.jboss.test.messaging.MessagingTestCase
                                Provider specfic Subclasses of Abstract class org.jboss.test.messaging.util.AdminUtility : responsible for getting the initial context, binding /unbinding of the administered objects.

                                Property file decides which subclass of this AdminUtility gets picked up.Sub class instantiated using reflection to keep the code independent of teh actual impl

                                e.g
                                org.jboss.test.messaging.util.provider.JBossMessAdminUtility extends
                                org.jboss.test.messaging.util.AdminUtility

                                JBossMessAdminUtility uses ServerWrapper internally(for the time being)

                                org.jboss.test.messaging.util.provider.JBossMQAdminUtility : for JBossMQ..



                                Let me know if this is ok,.
                                {Will come out with the design proposal for the perofrmance test framework in the other thread.}

                                • 14. Re: Can we start writing jms test clients to check complianc
                                  Ovidiu Feodorov Master


                                  An observation related to naming: org.jboss.test.messaging.util is intendend to contain tests for Messaging utility classes, not utilities. Please use org.jboss.test.messaging.tools for that.

                                  1 2 Previous Next