10 Replies Latest reply on Apr 19, 2011 10:55 AM by clebert.suconic

    Message endpoint in use?

    pinkushn

      I am a relative newcomer to JBoss.  I run JBoss on a linux vps with 720 megs of ram.  I have a fairly stable enterprise app that uses jms for two queues and four topics.  Up until this weekend, I was using AS 5.0.1 and default messaging.  This weekend, I upgraded to 5.1.0 and I also switched to hornet CR2. In very low load tests (one client) my new setup worked great, but when I tested with 20 live clients, I frequently got an error message in server.log that I have not seen before (despite having run successfully with about 40 clients in the old setup):

       

      java.lang.IllegalStateException: This message endpoint + StudentMDB is already in use by another thread Thread[Thread-49 (group:HornetQ-client-global-threads-3634025),5,]

                      at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:117)

                      at $Proxy171.beforeDelivery(Unknown Source)

                      at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:215)

                      at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:767)

                      at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:45)

                      at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:885)

                      at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)

                      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

                      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

                      at java.lang.Thread.run(Thread.java:619)


       

      I would appreciate any ideas that could point me in the right direction.

       

      Thanks,

       

      Josh Britton

        • 1. Re: Message endpoint in use?
          timfox

          If you can add a JIRA with exact instructions on how to reproduce, someone will take a look.

          • 2. Re: Message endpoint in use?
            pinkushn

            Thanks: I'll see if I can do that post-holidays.

             

            josh

            • 3. Re: Message endpoint in use?
              timfox

              Hello Josh-

               

              Did you get anywhere in replicating this?

              • 4. Re: Message endpoint in use?
                pinkushn

                Thanks for following up.  I finally got back to it today (holidays were long and great!).  I restarted from scratch with CR3, and now I no longer get the error.  Hornetq is performing smoothly.

                 

                josh

                • 5. Re: Message endpoint in use?
                  ataylor
                  Ok, thanks for that Josh.
                  • 6. Re: Message endpoint in use?
                    omerzohar

                    im using jboss as 6 + hornetq 2.1.2 final and under heavy load im still getting these errors:

                     

                     

                    2011-04-02 23:58:09,435 ERROR [org.hornetq.ra.inflow.HornetQMessageHandler] Failed to deliver message: java.lang.IllegalStateException: This message endpoint + FetcherMessageBean is already in use by another thread Thread[Thread-1641 (group:HornetQ-client-global-threads-29744243),5,]

                            at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:118) [:1.7.17]

                            at $Proxy229.beforeDelivery(Unknown Source)     at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:253) [:6.0

                    .0.Final]

                            at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:822) [:6.0.0.Final]

                            at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:46) [:6.0.0.Final]

                            at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:940) [:6.0.0.Final]

                            at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100) [:6.0.0.Final]

                            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [:1.6.0_20]

                            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [:1.6.0_20]

                            at java.lang.Thread.run(Thread.java:619) [:1.6.0_20]

                     

                    I think this problem starts with memory limit problems, but then even when the server is not loaded the this thread is not being released and every call to that bean gets this error.

                    • 7. Re: Message endpoint in use?
                      domenicofe

                      Hi all,

                      I have the same problem, quoted the Stacktrace, Any Idea how to troubleshoot it?

                       

                      16:42:43,336 ERROR [org.hornetq.ra.inflow.HornetQMessageHandler] Failed to deliver message: java.lang.NullPointerException

                                at org.jboss.ejb3.EJBContainer.pushEnc(EJBContainer.java:580) [:1.7.17]

                                at org.jboss.ejb3.mdb.MessagingContainer.localInvoke(MessagingContainer.java:300) [:1.7.17]

                                at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.delivery(MessageInflowLocalProxy.java:299) [:1.7.17]

                                at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:152) [:1.7.17]

                                at $Proxy136.onMessage(Unknown Source)          at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:256) [:6.0.0.Final]

                                at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:822) [:6.0.0.Final]

                                at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:46) [:6.0.0.Final]

                                at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:940) [:6.0.0.Final]

                                at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100) [:6.0.0.Final]

                                at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [:1.6.0_23]

                                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [:1.6.0_23]

                                at java.lang.Thread.run(Thread.java:662) [:1.6.0_23]

                      • 8. Re: Message endpoint in use?
                        clebert.suconic

                                  at org.jboss.ejb3.EJBContainer.pushEnc(EJBContainer.java:580) [:1.7.17]

                         

                         

                         

                        That's an EJB issue for sure.

                         

                         

                        What version of JBoss / HornetQ are you using?

                        • 9. Re: Message endpoint in use?
                          domenicofe

                          I'm using Jboss 6.0.0.Final with embedded HornetQ (I suppose 2.1.x) on Windows 7 64 bit (developer machine).

                          The test environment is:

                          1) Queue's Producer on a Singleton EJB

                          2) MDB's that insert consumed messages on PostgreSQL DS

                           

                          The Problem is randomly  happens.

                          Rate 25msg/sec

                          • 10. Re: Message endpoint in use?
                            clebert.suconic

                            Can  you switch to the  nightly build on AS6 and verify if that still happening?

                             

                            I'm not aware of any issues like that on 2.2