1 2 3 4 Previous Next 59 Replies Latest reply on Jul 18, 2009 7:44 AM by nbhatia Go to original post
      • 30. Re:
        ataylor

        Naresh,

        Ive checked in a couple of tweaks to trunk. any chance you could check it out (http://anonsvn.jboss.org/repos/messaging/trunk/), build it (ant distro) and try with this.

        Andy

        • 31. Re:
          nbhatia

          Ok I checked out your latest tweaks on the trunk, and it seems like the problem is gone. 10000 messages sent and received without any hiccups!. That's great!

          As a side note, it took 12 minutes for the full round trip, which translates to about 14 tps. Any tips on how this can be improved? Of course this is on my crappy laptop, but I will try it later on my new desktop, which has the Intel i7 processor.

          Thanks again for nailing this issue.

          Naresh

          • 32. Re:
            clebert.suconic

            As you are using MDBs, you are opening one XA transaction for each message received, what means several round trips for each message received.

            I also realized what you are saying, and as a result of running your testcase, I talked to Tim Fox this morning and opened this jira with a feature request:

            https://jira.jboss.org/jira/browse/JBMESSAGING-1686

            ATM if you don't use MDBs and batch your transactions would give you a much better performance.

            • 33. Re:
              nbhatia

              Please don't ask me to drop MDBs :-). I have tinkered with so many options since last month (including ActiveMQ), and settled on this one - at least it is working! I figured this is the best way to get robust connection pooling. Based on this, I even gave up my easier war packaging for an ear. So hope there's a better way to get MDBs performing - may be drop XA transactions and do some tricks like commit DB and then commit JMS.

              Thanks.
              Naresh

              • 34. Re:
                clebert.suconic

                Yes.. we will provide a feature to fix that issue.

                And your input will be very appreciated when we are developing it... what should happen pretty soon I believe.

                • 35. Re:
                  timfox

                   

                  "nbhatia" wrote:
                  Ok I checked out your latest tweaks on the trunk, and it seems like the problem is gone. 10000 messages sent and received without any hiccups!. That's great!

                  As a side note, it took 12 minutes for the full round trip, which translates to about 14 tps. Any tips on how this can be improved? Of course this is on my crappy laptop, but I will try it later on my new desktop, which has the Intel i7 processor.

                  Thanks again for nailing this issue.

                  Naresh


                  We know that JBM 2 can do many 1000s of msgs/sec even on a crappy laptop. Probably 99.9% of the time taken in your test is being spent in non JBM code.

                  Remember you're using MDBs and the JCA layer, both of which are parts of the AS (not part of JBM). You're probably opening and processing a JTA (XA) transaction for *every* message being consumed (JTA is heavyweight), again the JTA impl (JBoss TS) is part of the AS not JBM. JTA will require syncing to disc at several points in the 2PC commit protocol.

                  Then if you are doing other stuff in your transaction (maybe writing to a DB) you have the time taken writing, preparing, and committing the tx - that's processing in the DB, not JBM.

                  And finally you've got time processing your actual user code in your MDB onMessage.

                  • 36. Re:
                    nbhatia

                    Well, my test is very slim - it does not do any database transactions or message processing. I even disabled the logging of the messages to make sure I am getting raw message times. The only thing that is potentially slowing down the performance is the use of XA, through the use of JCA and JmsXA connection factory, I assume. I am completely flexible on dealing with this provided the JBM and JBoss AS teams jointly recommend the right way to do it.

                    As mentioned in my previous message, I have seen people abandon XA and 2PC in favor of performance. Usually they will commit the database transaction first, if that succeeds they will acknowledge the JMS message (this assumes that the DB commit will fail more often than a JMS ack). May be you can confirm or deny this as a best practice and recommend a solution using this approach. In any case, as an application developer, I need a holistic solution that plays well with JBoss AS and JBM. Hope you can help with this.

                    • 37. Re:
                      timfox

                      If you post your example application on a JIRA, then someone next week can have a look at it and see why it's acting slow...

                      In the mean-time - gave you looked at the performance tuning section in the jbm user manual?

                      • 38. Re:
                        timfox

                        In my experience most very high performance messaging installations are not using JEE (no MDBs, JTA, JCA) - all these things have an overhead and often you can design them out in your application.

                        E.g. you can often design out JTA by using duplicate detection (see user manual)

                        • 39. Re:
                          clebert.suconic

                           

                          "timfox" wrote:
                          If you post your example application on a JIRA, then someone next week can have a look at it and see why it's acting slow...


                          The test we were using to replicate the ping issue replicates this transaction issue.

                          • 40. Re:
                            timfox

                             

                            "clebert.suconic@jboss.com" wrote:
                            "timfox" wrote:
                            If you post your example application on a JIRA, then someone next week can have a look at it and see why it's acting slow...


                            The test we were using to replicate the ping issue replicates this transaction issue.


                            Yes I know, but I am not sure whether that is the exact same program that the user used to get 14 TPS (which seems extremely slow), that's why I asked for it.

                            • 41. Re:
                              nbhatia

                              My example is already posted in JIRA (see
                              https://jira.jboss.org/jira/browse/JBMESSAGING-1677 - JmsTest-v2.zip). It obviously uses all the bad things :-), i.e. MDB, JCA, JTA. I think the best thing to do is to change the example to work in the "recommended way" and use it as a reference.

                              I read the performance section in the manual. The most applicable one here seems to be "Re-use connections / sessions / consumers / producers". Indeed JCA is supposed to do that and hence every place I have read, the recommendation is to go with JCA. So, I am a bit confused - are you recommending to drop JCA and write our own connection pooling?

                              I also read the comment regarding not to use Spring JMS Template. Two questions here:

                              1) The JMS template is used only for sending messages and as long as we connect it to a pooled connection factory, there should be no problems, correct? Now how to get that pooled connection factory is a problem if the recommendation is not to use JCA.

                              2) The JMS template does not come into picture when receiving messages. Spring provides a MDP container which does its own pooling. So that should not be a problem either, correct?

                              So please help me understand how I can practically use the tuning recommendations. If I shouldn't be using JCA/JTA etc., then I have to manage resources myself. This means all application developers will write their own pooling code, which seems counter productive. Or we can use whatever is available out there (like Spring Template and MDP) and adapt it to work the right way.

                              Thoughts?

                              • 42. Re:
                                nbhatia

                                 

                                "timfox" wrote:
                                "clebert.suconic@jboss.com" wrote:
                                "timfox" wrote:
                                If you post your example application on a JIRA, then someone next week can have a look at it and see why it's acting slow...


                                The test we were using to replicate the ping issue replicates this transaction issue.


                                Yes I know, but I am not sure whether that is the exact same program that the user used to get 14 TPS (which seems extremely slow), that's why I asked for it.


                                Yes it is the exact same program.

                                • 43. Re:
                                  timfox

                                  I'm slightly confused - I can't see where in your test program the Spring JMS Template is being used...

                                  • 44. Re:
                                    timfox

                                    The "right" way depends on what you are trying to do.

                                    From your test program I can see you are just sending messages to a queue, and consuming them via an MDB which does nothing but log a message.

                                    In this case you don't need transactions on your MDB, also there's no real JCA involved here, since you're just sending messages from a client - there's nothing to pool.

                                    Also you are currently using persistent messages, for the purposes of this example that's unnecessary too.