1 2 Previous Next 20 Replies Latest reply on Sep 4, 2003 3:34 AM by gavinandrews

    3.2.2RC3, Fiorano, MDB, concurrency

    gavinandrews

      I have happily implemented a MDB under FioranoMQ 5.21 with 3.2.2RC3; it works when concurrency is low... i.e. send a message, bean gets onMessage, it is processed and disappears from the queue.

      However, if a message arrives before the first has been processed it appears that it is not picked up, indeed subsequent message arrivals do not kick the MDB to process them...

      Is there anything specific I should investigate?

      Thanks in advance,
      Gavin

        • 1. Re: 3.2.2RC3, Fiorano, MDB, concurrency

          Start with looking for errors in server.log
          Enabling TRACE debugging for org.jboss.ejb and org.jboss.jms
          might show more information?

          Also check for a deadlock by dumping the threads.

          Regards,
          Adrian

          • 2. Re: 3.2.2RC3, Fiorano, MDB, concurrency
            gavinandrews

            I turned debugging on; It just seems that after a bit of concurrency the onMessage of my MDBs are not being invoked.

            The problem is easily reproduceable; just take a trivial MDB and put some debugging and a Thread.Sleep in the onMessage. Then hammer the server with lots of messages; after a while it gets stuck and despite the JMS queue having work; JBoss no longer takes the messages off the queue.

            I have tried all combinations of Bean/Container and Required/NotSupported but get the same behaviour.

            processing can be resumed by stopping and restarting the JMSProvider..

            As I say I'm using FioranoMQ 5.21 but I can't see why this behaviour would necessarily be Provider specific?

            Anything else I can try?

            I looked in JMSContainerInvoker and MessageDrivenMetaData but I can't see anything obvious... the comment about ClientAcknowledge seems a odd but I don't think that's my problem.

            Regards,
            Gavin




















            • 3. Re: 3.2.2RC3, Fiorano, MDB, concurrency

              The delivery goes something like:

              MDB Container -> Fiorano ConnectionConsumer to get message
              MDB Conatiner -> ServerSessionPool -> FioranoSession
              FioranoSession -> JBoss MDB invoker -> your MDB

              Now the question is where is it getting stuck?
              Can you post a threaddump and the log?

              Regards,
              Adrian

              • 4. Re: 3.2.2RC3, Fiorano, MDB, concurrency
                gavinandrews

                I don't have a thread dump or log...

                I found a potential difference in how I did things before using JBOSS MDB... previously I have created non-transacted QueueConnections.

                I see from JMSContainerInvoker that the pool is always created as transacted (there is 'true' hardcoded)...

                Remember this is FioranoMQ 5.21 so all my sessions are non-XA.

                I wonder if this has something to do with it? Incidentally how do you turn trace on to see when the sessions are pullled from the pool?

                Thanks,
                Gavin

                • 5. Re: 3.2.2RC3, Fiorano, MDB, concurrency

                  It should detect automatically that it is not XA
                  You should have the following warning in your log:
                  log.warn("Using a non-XA TopicConnection. " +
                  "It will not be able to participate in a Global UOW");

                  There is an example for TRACE in conf/log4j.xml

                  Regards,
                  Adrian

                  • 6. Re: 3.2.2RC3, Fiorano, MDB, concurrency
                    gavinandrews

                    I do get the...

                    "Using a non-XA Queue Connection. It will not..."

                    messages; and the sessions are being created in the pool...

                    "added server session to the pool"

                    What I am suspicious about is the .createQueueSession(transacted, ack) since transacted is set true from JMSContainerInvoker and in this case I don't have a transaction.

                    Regards,
                    Gavin

                    • 7. Re: 3.2.2RC3, Fiorano, MDB, concurrency

                      It is a JMS client transaction, nothing to do with JTA.

                      Regards,
                      Adrian

                      • 8. Re: 3.2.2RC3, Fiorano, MDB, concurrency
                        gavinandrews

                        >>It is a JMS client transaction, nothing to do with JTA.
                        I can see that.

                        I'll turn on tracing and try it again...

                        Incidentally... stopDelivery, then start and then startDelivery on the bean gets things going again... for a while...

                        Regards,
                        Gavin


                        • 9. Re: 3.2.2RC3, Fiorano, MDB, concurrency
                          gavinandrews

                          I had a brainwave...

                          Instead of being related to concurrency this issue is simply competition for the StdJMSPool. I then reduced the StdJMSPool MaxSize to 2 and started JBOSS.

                          The first message is consumed correctly...
                          The second message is consumed correctly...

                          and the 3rd hangs the server.

                          Looks like sessions are not being returned to the pool or aren't re-useable or something like that...?

                          Regards,
                          Gavin

                          • 10. Re: 3.2.2RC3, Fiorano, MDB, concurrency
                            gavinandrews

                            Yeap; that's it...

                            The first message works ok...
                            The second message works ok...

                            Then when the third message arrives I get

                            2003-09-03 09:52:00,599 TRACE [org.jboss.jms.asf.StdServerSessionPool] getting a server session

                            and then it stops... no more messages... no nothing.

                            Regards,
                            Gavin

                            • 11. Re: 3.2.2RC3, Fiorano, MDB, concurrency
                              gavinandrews

                              Here is a slightly more complete trace... i.e. it shows the 2nd message arriving and being serviced...


                              2003-09-03 09:51:20,146 TRACE [org.jboss.jms.asf.StdServerSessionPool] getting a server session
                              2003-09-03 09:51:20,146 TRACE [org.jboss.jms.asf.StdServerSessionPool] using server session: org.jboss.jms.asf.StdServerSession@1d652
                              2003-09-03 09:51:20,146 TRACE [org.jboss.jms.asf.StdServerSession] starting invokes on server session
                              2003-09-03 09:51:20,146 TRACE [org.jboss.jms.asf.StdServerSession] onMessage running (pool, session, xaSession, useLocalTX): , fiorano.jms.runtime.ptp.FioranoQueueSession@19e3e24, null, false
                              2003-09-03 09:51:20,146 TRACE [org.jboss.ejb.plugins.jms.JMSContainerInvoker] processing message: fiorano.jms.services.msg.def.FioranoTextMessage@4d6b1a
                              2003-09-03 09:51:20,146 TRACE [org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor] invokerBInding is null in ProxyFactoryFinder
                              2003-09-03 09:51:20,146 TRACE [org.jboss.ejb.plugins.LogInterceptor] Start method=onMessage
                              2003-09-03 09:51:20,146 TRACE [org.jboss.ejb.plugins.MessageDrivenInstancePool] Get instance org.jboss.ejb.plugins.MessageDrivenInstancePool@11c135c#1#class MDB
                              2003-09-03 09:51:20,146 INFO [STDOUT] >onMessage
                              2003-09-03 09:51:20,146 INFO [STDOUT] Bean got message from onMessage():09:51:20 Test
                              2003-09-03 09:51:20,146 INFO [STDOUT] Starting Wait...
                              2003-09-03 09:51:20,161 TRACE [org.jboss.jms.asf.StdServerSession] running...
                              2003-09-03 09:51:22,146 INFO [STDOUT] Finished Waiting
                              2003-09-03 09:51:22,146 INFO [STDOUT] <onMessage
                              2003-09-03 09:51:22,146 TRACE [org.jboss.ejb.plugins.MessageDrivenInstancePool] 0/100 Free instance:org.jboss.ejb.plugins.MessageDrivenInstancePool@11c135c#null#null#true#class MDB
                              2003-09-03 09:51:22,146 TRACE [org.jboss.ejb.plugins.LogInterceptor] End method=onMessage
                              2003-09-03 09:51:22,146 TRACE [org.jboss.jms.asf.StdServerSession] Commiting the JMS transaction
                              2003-09-03 09:51:22,146 TRACE [org.jboss.jms.asf.StdServerSession] onMessage done

                              2003-09-03 09:51:40,489 TRACE [org.jboss.ejb.plugins.LRUEnterpriseContextCachePolicy] Resized cache for bean InstrumentPrice: old capacity = 1000000, new capacity = 50

                              <<<<<<<<< 3rd message arrives

                              2003-09-03 09:52:00,599 TRACE [org.jboss.jms.asf.StdServerSessionPool] getting a server session

                              2003-09-03 09:52:10,411 TRACE [org.jboss.ejb.plugins.LRUEnterpriseContextCachePolicy] Resized cache for bean BorisOrderProperty: old capacity = 1000000, new capacity = 50


                              I can't see the sessions being recycled into StdJMSPool; which I guess explains why they get used up and then it it gets blocked waiting on the SessionPool.

                              Regards,
                              Gavin

                              • 12. Re: 3.2.2RC3, Fiorano, MDB, concurrency

                                So the Fiorano session is not returning from its run() operation?

                                Regards,
                                Adrian

                                • 13. Re: 3.2.2RC3, Fiorano, MDB, concurrency
                                  gavinandrews

                                  That appears to be it.

                                  Fiorano's 5.21 implementation of..

                                  javax.jms.Session.run for the created QueueSession does not apper to return once all outstanding messages have been sent to the session and hence
                                  the StdServerSession.run doesn't move on and hence the session is not recycled.

                                  Not sure what to do now... might just go and bang my head against a wall.

                                  Regards,
                                  Gavin

                                  • 14. Re: 3.2.2RC3, Fiorano, MDB, concurrency

                                    Take a threaddump (ctrl-break windows, ctrl-\ on unix)
                                    Look for deadlocks.

                                    Regards,
                                    Adrian

                                    1 2 Previous Next