1 Reply Latest reply on Jan 14, 2010 1:26 PM by adrian.brock

    Slow dequeue for singleton MDB

    mohitanchlia

      We use tibco ems and Jboss 4.2 CP 6. For the queues in question:

       

      1. Every onMessage calls take around 10-20ms.
      2. According to the stats we should be able to process around 50 messages per sec. worst case.
      3. But we are only able to do 15 messages per sec.

       

      So there is a delay and messages are not being sent. It gets worse as load increases. CPU on tibco ems server is only 96% idle and no iowaits. CPU on Jboss server is around 90% idle. It looks like a performance issue with Jboss fetching the messages. I have also tried setting "prefetch" in tibco ems. It looks like Jboss is slow when dequeing messages for singletons. Even though MDB is available it doesn't get message fast enough.

       

      Could someone please help me performance tuning or where the issue might be.

       

      We tried using JBossJMSMessageEndpointFactory and have followed what's in http://community.jboss.org/wiki/JBMPerf with no success.

        • 1. Re: Slow dequeue for singleton MDB

          This isn't really a JBoss Messaging question, if you're using Tibco EMS.

           

          Anyway, the short answer is that we can't really answer your question, you'll have to ask Tibco.

           

          The longer answer is that the delivery and prefetching of messages to an MDB is controlled by the ConnectionConsumer

          which is implemented by Tibco (which is why only they can answer the question).

          http://java.sun.com/j2ee/1.4/docs/api/javax/jms/ConnectionConsumer.html

           

          The only thing we control is the ServerSessionPool which you've set to one Session if you

          are using a Singleton MDB.

           

          Their ConnectionConsumer will have some code that does something like the following:


          while (notClosed)
          {
             Message m = getNextMessageFromServer(); // A
             ServerSession s = jbossServerSessionPool.getServerSession(); // B
             addMessageToSession(s, m);
             s.start(); // C
          }
          

           

          Now at point (C) above we offload the work to a different thread so it ought to be able to get the next message from (A)

          while it is processing the previous message.

          But until that previous message completes, point (B) will block - that is the purpose of the singleton - no concurrent invocation.


          Once the previous work has finished, point (B) will unblock and it should immediately process the next message.

          Whether that is true, depends upon how Tibco implement the above method. e.g. if they have (A) and (B) the other way around, it

          isn't going to get the next message until a session is available, which means you will see a latency.

           

          NOTE: You can't just time your MDB's onMessage() method. There's more work than that (assuming you are using JTA transactions).

          We wrap your onMessage() method with something like:

           

          Transasction tx = startTransaction();
          tx.enlistResource(tibcoXAResource);
          try
          {
             yourMDB.onMessage(message);
          }
          finally
          {
             tx.commit/rollback();
          }
          

           

          If you are doing work in a database in your MDB, the commit() at the end will have to do a 2 phase commit,

          one branch for the DB and one for the Tibco message acknowledgement.