14 Replies Latest reply on Aug 22, 2007 7:00 PM by Clebert Suconic

    ERROR: Cannot find delivery to cancel

    Wojciech Kuranowski Newbie

      Hi.

      Tonight our partner had connection problems and our MDB couldn't connect to their server. We have rather long connection timeout - 40s.

      In logs, I can find some errors. Is this a bug in JBM? Or maybe something else?

      We are testing our appplication with JBoss 4.2.1.GA and JBM 1.4.0.CR1 with MySQL persistence and InnoDB tables.

      2007-07-24 01:15:47,633 ERROR [org.jboss.messaging.util.ExceptionUtil] SessionEndpoint[z-687b7h4f-1-mnu30h4f-djh1nc-e600a] cancelDelivery [w2k1-5ruukh4f-1-mnu30h4f-djh1nc-e600a]
      javax.jms.IllegalStateException: Cannot find delivery to cancel 3304
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.cancelDeliveryInternal(ServerSessionEndpoint.java:1616)
       at org.jboss.jms.server.endpoint.ServerSessionEndpoint.cancelDelivery(ServerSessionEndpoint.java:390)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised.org$jboss$jms$server$endpoint$advised$SessionAdvised$cancelDelivery$aop(SessionAdvised.java:138)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised$cancelDelivery_2980985335510575813.invokeNext(SessionAdvised$cancelDelivery_2980985335510575813.java)
       at org.jboss.jms.server.container.ServerLogInterceptor.invoke(ServerLogInterceptor.java:105)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised$cancelDelivery_2980985335510575813.invokeNext(SessionAdvised$cancelDelivery_2980985335510575813.java)
       at org.jboss.jms.server.endpoint.advised.SessionAdvised.cancelDelivery(SessionAdvised.java)
       at org.jboss.jms.wireformat.SessionCancelDeliveryRequest.serverInvoke(SessionCancelDeliveryRequest.java:84)
       at org.jboss.jms.server.remoting.JMSServerInvocationHandler.invoke(JMSServerInvocationHandler.java:165)
       at org.jboss.remoting.ServerInvoker.invoke(ServerInvoker.java:734)
       at org.jboss.remoting.transport.local.LocalClientInvoker.invoke(LocalClientInvoker.java:101)
       at org.jboss.remoting.Client.invoke(Client.java:1550)
       at org.jboss.remoting.Client.invoke(Client.java:530)
       at org.jboss.remoting.Client.invoke(Client.java:518)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:186)
       at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:157)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$cancelDelivery$aop(ClientSessionDelegate.java:461)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$cancelDelivery_2980985335510575813.invokeNext(ClientSessionDelegate$cancelDelivery_2980985335510575813.java)
       at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:91)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$cancelDelivery_2980985335510575813.invokeNext(ClientSessionDelegate$cancelDelivery_2980985335510575813.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$cancelDelivery_2980985335510575813.invokeNext(ClientSessionDelegate$cancelDelivery_2980985335510575813.java)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.cancelDelivery(ClientSessionDelegate.java)
       at org.jboss.jms.client.container.ClientConsumer.checkExpiredOrReachedMaxdeliveries(ClientConsumer.java:103)
       at org.jboss.jms.client.container.ClientConsumer.callOnMessage(ClientConsumer.java:132)
       at org.jboss.jms.client.container.SessionAspect.handleRun(SessionAspect.java:801)
       at org.jboss.aop.advice.org.jboss.jms.client.container.SessionAspect14.invoke(SessionAspect14.java)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$run_N8003352271541955702.invokeNext(ClientSessionDelegate$run_N8003352271541955702.java)
       at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
       at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
       at org.jboss.jms.client.delegate.ClientSessionDelegate$run_N8003352271541955702.invokeNext(ClientSessionDelegate$run_N8003352271541955702.java)
       at org.jboss.jms.client.delegate.ClientSessionDelegate.run(ClientSessionDelegate.java)
       at org.jboss.jms.client.JBossSession.run(JBossSession.java:199)
       at org.jboss.resource.adapter.jms.inflow.JmsServerSession.run(JmsServerSession.java:237)
       at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:204)
       at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:275)
       at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:743)
       at java.lang.Thread.run(Thread.java:595)
      
      


        • 1. Re: ERROR: Cannot find delivery to cancel
          Tim Fox Master

          Can you please give instructions on how to replicate?

          • 2. Re: ERROR: Cannot find delivery to cancel
            Wojciech Kuranowski Newbie

             

            "timfox" wrote:
            Can you please give instructions on how to replicate?


            1. Queue with RedeliveryDelay = 5000
            2. MDB

            config:
            @MessageDriven(activationConfig = {
             @ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"),
             @ActivationConfigProperty(propertyName="destination", propertyValue="queue/queueName"),
             @ActivationConfigProperty(propertyName="maxSession", propertyValue="1")
            })
            


            onMessage():
            try {
             SocketAddress sockAddr = new InetSocketAddress(InetAddress.getByName(123.123.123.123), 1234);
             Socket sock = new Socket();
             sock.setSoTimeout(0);
             sock.setTcpNoDelay(true);
             sock.connect(sockAddr, 40000);
            } catch (Throwable te) {
             log.error("Stack trace:", te);
             mdc.setRollbackOnly();
            }
            


            3. Stateless Bean with xxx() method which inserts something into the queue.
            4. Client which calls xxx() in a loop:
             TestJBM t = (TestJBM) (new InitialContext()).lookup("TestJBMBean/remote");
             while (true) {
             t.xxx();
             Thread.sleep(20000);
             }
            


            After 2 minutes I have errors mentioned in a first post.


            • 3. Re: ERROR: Cannot find delivery to cancel
              Tim Fox Master

              Thanks Noxis-

              We'll take a look at this next week.

              • 4. Re: ERROR: Cannot find delivery to cancel
                Wojciech Kuranowski Newbie

                 

                "timfox" wrote:
                We'll take a look at this next week.


                Can you reproduce the problem?

                PS. When will you release JBM 1.4.0.GA? Road Map is not accurate at the moment :)


                • 5. Re: ERROR: Cannot find delivery to cancel
                  Tim Fox Master

                  Yes, we are very busy.

                  We'll get around to it eventually.

                  If you want to lend a hand you would be most welcome. I have plenty of JIRA tasks I could assign to you?

                  • 7. Re: ERROR: Cannot find delivery to cancel
                    Tim Fox Master

                    I have followed your instructions and tried to replicate this with AS 4.2.0.GA (CR1 is only tested with 4.2.0), but I cannot replicate.

                    Can you verify if the problem occurs with 4.2.0? Then I can mark as cannot replicate.

                    • 8. Re: ERROR: Cannot find delivery to cancel
                      Wojciech Kuranowski Newbie

                       

                      "timfox" wrote:
                      I have followed your instructions and tried to replicate this with AS 4.2.0.GA (CR1 is only tested with 4.2.0), but I cannot replicate.

                      Can you verify if the problem occurs with 4.2.0? Then I can mark as cannot replicate.


                      I am able to replicate this bug also on AS 4.2.0.GA + JBM 1.3.0.GA.
                      Can you send me your full sources? I will try to replicate here.


                      • 9. Re: ERROR: Cannot find delivery to cancel
                        Tim Fox Master

                        I just copied what you posted.

                        If you can package up a test program and exact step by step instructions I will try again

                        • 10. Re: ERROR: Cannot find delivery to cancel
                          Wojciech Kuranowski Newbie

                           

                          "timfox" wrote:
                          I just copied what you posted.

                          If you can package up a test program and exact step by step instructions I will try again


                          I think I forgot about one important thing - TTL of messages.

                          Here is a working example. I can replicate this bug all the time. "Cannot find delivery to cancel" appears after about 2 minutes.

                          Queue configuration:
                          <?xml version="1.0" encoding="UTF-8"?>
                          <server>
                           <mbean code="org.jboss.jms.server.destination.QueueService" name="jboss.messaging.destination:service=Queue,name=testCancel" xmbean-dd="xmdesc/Queue-xmbean.xml">
                           <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
                           <depends>jboss.messaging:service=PostOffice</depends>
                           <attribute name="RedeliveryDelay">5000</attribute>
                           </mbean>
                          </server>
                          


                          TestMDB.java
                          package main.ejb;
                          
                          import javax.annotation.Resource;
                          import javax.ejb.MessageDriven;
                          import javax.ejb.ActivationConfigProperty;
                          import javax.ejb.MessageDrivenContext;
                          import javax.jms.Message;
                          import javax.jms.MessageListener;
                          import javax.jms.TextMessage;
                          
                          @MessageDriven(activationConfig = {
                           @ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"),
                           @ActivationConfigProperty(propertyName="destination", propertyValue="queue/testCancel"),
                           @ActivationConfigProperty(propertyName="maxSession", propertyValue="1")
                           })
                          
                          public class TestMDB implements MessageListener {
                          
                           @Resource
                           private MessageDrivenContext mdc;
                          
                           public void onMessage(Message msg) {
                           try {
                           TextMessage textMsg = (TextMessage) msg;
                           System.out.println("got: " + textMsg.getText());
                           Thread.sleep(40000);
                           mdc.setRollbackOnly();
                           } catch (Throwable te) {
                           te.printStackTrace();
                           mdc.setRollbackOnly();
                           }
                           }
                          }
                          



                          Test.java (interface)
                          package main.ejb;
                          
                          import javax.ejb.Remote;
                          
                          @Remote
                          public interface Test {
                          
                           void testCancel();
                          }
                          


                          TestBean.java
                          package main.ejb;
                          
                          import javax.annotation.Resource;
                          import javax.ejb.Stateless;
                          import javax.jms.Connection;
                          import javax.jms.ConnectionFactory;
                          import javax.jms.JMSException;
                          import javax.jms.MessageProducer;
                          import javax.jms.Queue;
                          import javax.jms.Session;
                          import javax.jms.TextMessage;
                          
                          @Stateless
                          public class TestBean implements Test {
                          
                           @Resource(mappedName = "ConnectionFactory")
                           private ConnectionFactory factory;
                          
                           @Resource(mappedName = "queue/testCancel")
                           private Queue queue;
                          
                           public void testCancel() {
                           Connection connection = null;
                           Session session = null;
                           MessageProducer producer = null;
                           try {
                           connection = factory.createConnection();
                           session = connection.createSession(true, 0);
                           producer = session.createProducer(queue);
                          
                           TextMessage msg = session.createTextMessage();
                           msg.setText("just another test");
                          
                           producer.setTimeToLive(90000);
                          
                           producer.send(msg);
                           session.commit();
                          
                           } catch (Throwable te) {
                           te.printStackTrace();
                           } finally {
                           try {
                           if (producer != null) {
                           producer.close();
                           }
                           if (session != null) {
                           session.close();
                           }
                           if (connection != null) {
                           connection.close();
                           }
                           } catch (JMSException e) {
                           e.printStackTrace();
                           }
                           }
                           }
                          }
                          


                          TestIt.java (Client)
                          package test;
                          
                          import javax.naming.InitialContext;
                          
                          import main.ejb.Test;
                          
                          public class TestIt {
                          
                           public static void main(final String[] args) throws Exception {
                           Test t = (Test) (new InitialContext()).lookup("TestBean/remote");
                           while (true) {
                           t.testCancel();
                           Thread.sleep(20000);
                           }
                           }
                          }
                          



                          • 11. Re: ERROR: Cannot find delivery to cancel
                            Tim Fox Master

                            Ok, I will try again this coming week.

                            • 12. Re: ERROR: Cannot find delivery to cancel
                              Clebert Suconic Master

                              Noxis,

                              I'm trying to replicate this.. and I had played with several different versions, different configurations and I couldn't replicate this problem.

                              Maybe you could modify the ejb3mdb example in such way to replicate this.. it would help a lot.


                              In your example you are playing with sockets.. maybe there is something wrong with your sockets also.. I would need to take a look on logs to verify what's going on (Maybe this is something specific to your environment?).

                              If you could replicate the problem in an independent manner it would help us a lot!

                              • 13. Re: ERROR: Cannot find delivery to cancel
                                Wojciech Kuranowski Newbie

                                 

                                "clebert.suconic@jboss.com" wrote:
                                Noxis,

                                In your example you are playing with sockets.. maybe there is something wrong with your sockets also..



                                You have tried a wrong example! :)
                                Please check my post in this thread from Fri Aug 17, 2007 08:06 AM
                                (full sources without using sockets)




                                • 14. Re: ERROR: Cannot find delivery to cancel
                                  Clebert Suconic Master

                                  duh!

                                  I have it replicated.. I have attached a pach on the JIRA issue with a test replicating.. I didn't need a Stateless Bean.

                                  Thanks a lot for your help!