7 Replies Latest reply on Jan 29, 2007 5:37 AM by bob_walker99

    Messaging NotFoundInDispatcherException

    guinsu
    guinsu Dec 28, 2006 1:50 PM

      I just switched from messaging 1.0.1CR5 to 1.0.1SP2 because the old version would consistently silently stop sending messages on certain queues under load. Now with SP2 I get an error on my client, but nothing on the server. After 20-60 minutes of running under load, I get the following exception on one of my clients. I'm not sure how to pull this out into a test case since it is a large code base, but any ideas on what could cause this?

      org.jboss.aop.NotFoundInDispatcherException: Object with oid: -2147483628 was not found in the Dispatcher
      at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:85)
      at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:127)
      at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:1014)
      at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:508)
      at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:645)
      at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:293)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:172)
      at org.jboss.remoting.Client.invoke(Client.java:607)
      at org.jboss.remoting.Client.invoke(Client.java:599)
      at org.jboss.jms.client.delegate.DelegateSupport.invoke(DelegateSupport.java:111)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledge_354868566256058116.invokeNext(ClientSessionDelegate$acknowledge_354868566256058116.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:182)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledge_354868566256058116.invokeNext(ClientSessionDelegate$acknowledge_354868566256058116.java)
      at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:69)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledge_354868566256058116.invokeNext(ClientSessionDelegate$acknowledge_354868566256058116.java)
      at org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$acknowledge_354868566256058116.invokeNext(ClientSessionDelegate$acknowledge_354868566256058116.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.acknowledge(ClientSessionDelegate.java)
      at org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:218)
      at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect6.invoke(SessionAspect6.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:182)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
      at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:69)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
      at org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
      at org.jboss.jms.client.remoting.MessageCallbackHandler.postDeliver(MessageCallbackHandler.java:162)
      at org.jboss.jms.client.remoting.MessageCallbackHandler.receive(MessageCallbackHandler.java:448)
      at org.jboss.jms.client.container.ReceiverAspect.handleReceive(ReceiverAspect.java:63)
      at org.jboss.aop.advice.org.jboss.jms.client.container.ReceiverAspect24.invoke(ReceiverAspect24.java)
      at org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:182)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
      at org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
      at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:69)
      at org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
      at org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
      at org.jboss.jms.client.delegate.ClientConsumerDelegate$receive_N8299950230150603585.invokeNext(ClientConsumerDelegate$receive_N8299950230150603585.java)
      at org.jboss.jms.client.delegate.ClientConsumerDelegate.receive(ClientConsumerDelegate.java)
      at org.jboss.jms.client.JBossMessageConsumer.receive(JBossMessageConsumer.java:86)
      at dealcatcher.kolinka.jms.JMSReceiver.receive(JMSReceiver.java:57)
      at dealcatcher.kolinka.indexer.IndexReceiver.receive(IndexReceiver.java:45)
      at dealcatcher.kolinka.indexer.IndexServer.run(IndexServer.java:120)

      • 4132 Views
      • Tags:
        • 1. Re: Messaging NotFoundInDispatcherException
          ovidiu.feodorov
          ovidiu.feodorov Jan 10, 2007 11:23 PM (in response to guinsu)

          Does it lock up? If it does, can you send me a thread dump?
          What JVM are you using on server?

            • Actions
          • 2. Re: Messaging NotFoundInDispatcherException
            bob_walker99
            bob_walker99 Jan 17, 2007 5:27 AM (in response to guinsu)

            Hi Ovidiu,
            I am also seeing this error on my client, but nothing in the server logs. It concides with another couple of errors, which I don't know if are related or not, I'll post the full stack trace below, but in short:

            "ERROR remoting.MessageCallbackHandler - Failed to execute more()"
            and
            "ERROR remoting.MessageCallbackHandler - Failed to deliver message
            javax.jms.IllegalStateException: Should only be one entry in list. There are 77"

            the 77 figure is incremental, the error reappears repeatedly with a higher figure each time.

            It doesn't lock up and I haven't got a console to request a thread dump I'm afraid. I am using the following JVM on the server ( which is SLES 9 on x86_64):
            "Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-b105, mixed mode)"


            INFO | jvm 1 | 2007/01/17 10:10:27 | 917736 [Thread-13] ERROR remoting.MessageCallbackHandler - Failed to deliver message
            INFO | jvm 1 | 2007/01/17 10:10:27 | javax.jms.IllegalStateException: Should only be one entry in list. There are 77
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.SessionAspect.handlePostDeliver(SessionAspect.java:206)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect6.invoke(SessionAspect6.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:182)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:69)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientSessionDelegate$postDeliver_5319211143798977162.invokeNext(ClientSessionDelegate$postDeliver_5319211143798977162.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientSessionDelegate.postDeliver(ClientSessionDelegate.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.remoting.MessageCallbackHandler.postDeliver(MessageCallbackHandler.java:166)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.remoting.MessageCallbackHandler.callOnMessage(MessageCallbackHandler.java:141)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.remoting.MessageCallbackHandler$ListenerRunner.run(MessageCallbackHandler.java:757)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at EDU.oswego.cs.dl.util.concurrent.QueuedExecutor$RunLoop.run(QueuedExecutor.java:89)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at java.lang.Thread.run(Unknown Source)
            INFO | jvm 1 | 2007/01/17 10:10:27 | 917830 [Thread-13] ERROR container.ExceptionInterceptor - Caught Exception:
            INFO | jvm 1 | 2007/01/17 10:10:27 | org.jboss.aop.NotFoundInDispatcherException: Object with oid: -2147483622 was not found in the Dispatcher
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:85)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:127)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:1014)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:508)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:645)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:293)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:172)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.Client.invoke(Client.java:630)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.Client.invoke(Client.java:622)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.DelegateSupport.invoke(DelegateSupport.java:111)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate$more_8808215012022788963.invokeNext(ClientConsumerDelegate$more_8808215012022788963.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:182)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate$more_8808215012022788963.invokeNext(ClientConsumerDelegate$more_8808215012022788963.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:69)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate$more_8808215012022788963.invokeNext(ClientConsumerDelegate$more_8808215012022788963.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate$more_8808215012022788963.invokeNext(ClientConsumerDelegate$more_8808215012022788963.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate.more(ClientConsumerDelegate.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.remoting.MessageCallbackHandler$ListenerRunner.run(MessageCallbackHandler.java:777)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at EDU.oswego.cs.dl.util.concurrent.QueuedExecutor$RunLoop.run(QueuedExecutor.java:89)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at java.lang.Thread.run(Unknown Source)
            INFO | jvm 1 | 2007/01/17 10:10:27 | 917830 [Thread-13] ERROR remoting.MessageCallbackHandler - Failed to execute more()
            INFO | jvm 1 | 2007/01/17 10:10:27 | org.jboss.jms.util.MessagingJMSException: Caught exception
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:99)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate$more_8808215012022788963.invokeNext(ClientConsumerDelegate$more_8808215012022788963.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ClientLogInterceptor.invoke(ClientLogInterceptor.java:107)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate$more_8808215012022788963.invokeNext(ClientConsumerDelegate$more_8808215012022788963.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate.more(ClientConsumerDelegate.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.remoting.MessageCallbackHandler$ListenerRunner.run(MessageCallbackHandler.java:777)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at EDU.oswego.cs.dl.util.concurrent.QueuedExecutor$RunLoop.run(QueuedExecutor.java:89)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at java.lang.Thread.run(Unknown Source)
            INFO | jvm 1 | 2007/01/17 10:10:27 | Caused by: org.jboss.aop.NotFoundInDispatcherException: Object with oid: -2147483622 was not found in the Dispatcher
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.aop.Dispatcher.invoke(Dispatcher.java:85)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:127)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:1014)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:508)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:645)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:293)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:172)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.Client.invoke(Client.java:630)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.remoting.Client.invoke(Client.java:622)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.DelegateSupport.invoke(DelegateSupport.java:111)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate$more_8808215012022788963.invokeNext(ClientConsumerDelegate$more_8808215012022788963.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:182)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:117)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.delegate.ClientConsumerDelegate$more_8808215012022788963.invokeNext(ClientConsumerDelegate$more_8808215012022788963.java)
            INFO | jvm 1 | 2007/01/17 10:10:27 | at org.jboss.jms.client.container.ExceptionInterceptor.invoke(ExceptionInterceptor.java:69)
            INFO | jvm 1 | 2007/01/17 10:10:27 | ... 7 more

              • Actions
            • 3. Re: Messaging NotFoundInDispatcherException
              bob_walker99
              bob_walker99 Jan 17, 2007 5:39 AM (in response to guinsu)

              I should mention - I have just recently started using Tanuki Software's Java Service wrapper on my client, which runs on a remote server. I previously had no console logging for that client; this is the first opportunity I have had to log console output and be able to study it in any detail. I therefore can't state categorically that this hasn't always happened.

              I've had what I *believe* to be a working system in place for a number of weeks now thanks to Tim and Clebert's help with another bug.

              So, I'm not convinced that this error is preventing my messages from getting through and the system from working, but obviously I thought I should bring them to your attention.

              Let me know if you need more info,

              Best regards,

              Bob

                • Actions
              • 4. Re: Messaging NotFoundInDispatcherException
                timfox
                timfox Jan 17, 2007 5:43 AM (in response to guinsu)

                Have you seen this? :
                http://www.jboss.com/index.html?module=bb&op=viewtopic&t=98859

                Are you sure you aren't sharing your JMS session between multiple threads?

                  • Actions
                • 5. Re: Messaging NotFoundInDispatcherException
                  bob_walker99
                  bob_walker99 Jan 17, 2007 7:44 AM (in response to guinsu)

                  I hadn't seen that - it's entirely possible - I'll check it out, thanks Tim.
                  Regards,
                  Bob

                    • Actions
                  • 6. Re: Messaging NotFoundInDispatcherException
                    bob_walker99
                    bob_walker99 Jan 17, 2007 9:46 AM (in response to guinsu)

                    Doh. I have now RTFM and amended my code. It's looking much better.
                    Sorry to pollute the forum. :-$
                    All the best, Bob

                      • Actions
                    • 7. Re: Messaging NotFoundInDispatcherException
                      bob_walker99
                      bob_walker99 Jan 29, 2007 5:37 AM (in response to guinsu)

                      I'm still seeing
                      "org.jboss.aop.NotFoundInDispatcherException: Object with oid: -2147471496 was not found in the Dispatcher"
                      in my logs, it seems to happen after a couple of days uptime, and as far as I can tell, always when trying to create a new Session from an existing Connection object.
                      Can you shed some light onto what the exception actually means so I can see if it is something I am doing wrong in my code?

                        • Actions