5 Replies Latest reply on Aug 23, 2012 10:10 AM by Clebert Suconic

    Fifo delivery fails sometimes

    Jan van de klok Newbie

      Hello there,

      Maybey someone can help me with this problem:

       

      We have a jboss 6.0.1 running with hornetq 2.2.14.

       

      Our application  takes in messages from numerous sources and puts these messages in certain queue's for certain destination. Each unique destiniation has 1 queue with 1 consumer. Theses queue's are FIFO.

       

      we now have had a situation where 2 masage came in 25 seconds after eachother.

      Both mesages have been but on the same destination queue.

      The first received massga has an jms id that is lower than the second message.

      Both receive processes are in separate threads.

       

      On rare occasions, the second mesage is consumed before the first message. There is only one consumer for this queue!!

       

      1) How is this possible??? 

       

      2) What determines the delivery sequence?

           Is it the moment where the message is put on the queue  or is it the moment when the queue session is committed???

       

      Does anyone has an answewr to these questions.

       

      Any help is appreciated, 

       

      Jan van de Klok

        • 1. Re: Fifo delivery fails sometimes
          Andy Taylor Master

          1) How is this possible???

          Its not, the caveat being with redelivery.

          2) What determines the delivery sequence?

               Is it the moment where the message is put on the queue  or is it the moment when the queue session is committed???

          messages are added to the queue in the order they are sent by a producer, a consumer consumes from a queue in the order they are in the queue

          Both receive processes are in separate threads.

          This could be your issue, firstly you shouldnt have multiple threads using the same session at the same time, also you are probably introducing some non deterministic behaviour, also remember messages are buffered by the consumer.

          • 2. Re: Fifo delivery fails sometimes
            Jan van de klok Newbie

            Hi Andy,

             

            Thanks for your response.

             

            We are not sharing the same session between threads. This is the code that handles the incomming messages:

             

              Session session = null;

                PooledQueueSession pooledQueueSession = null;

                Transaction databaseTx = null;

                try

                {

                  session = getSessionFactory().openSession();

                  databaseTx = session.beginTransaction(); // start the database transaction  

                  Set<DeliveryServiceData> destinations = determineDestinations(message.getOperation(),namespace.getNamespaceUrl(),message.getSender(), message.getDestinations());

                  pooledQueueSession = client.getPooledQueueSession();

                  for (DeliveryServiceData destination : destinations)

                  {

                    client.storeMessageOnQueue(namespace, session, pooledQueueSession, message.getXmlAsStringMessage(), response, destination.getQueueName(), destination.getReceiver(), message.getOperation(), message.getMessageId());

                  }

                  if (destinations.isEmpty())

                  {

                    throw new RuntimeException("No valid destination found to deliver messages check broker configuration for destination: " + message.getDestinations());

                  }

                  databaseTx.commit(); // commmit the database

                  pooledQueueSession.commit(); //commit the queues

                }

                catch (Exception e) ....

             

            We do have multiple threads that  recieve messages but each thread is using a queueSession from a "queueSessionPool", so each thread has its own session. In  that session it could however put messages in the same queue.

             

            That's why I was wondering:  what determines the position of the message in a queue, is it  the Put method or is it the commit of the session.??

             

            So in case of :

             

               thread1         put msgx in queue A

               thread2         put msgY in queue A

               commit thread2

               commit thread1

             

               Would the mesasage be consumed from the queue A in the order   or msgx, msgy    or msgy, msgx  ??  I think that's the big question here.

             

             

            Regards Jan

            • 3. Re: Fifo delivery fails sometimes
              Andy Taylor Master

              The message arrives at the queue when it is commited, but remember ordering is point to point, i.e. between 1 producer and 1 consumer, so even when you think messages are out of order they probably arent. Basically you can't determine order between multiple producers and consumers.

              • 4. Re: Fifo delivery fails sometimes
                Jan van de klok Newbie

                Ok Andy thanks. Your answers are helping me.

                • 5. Re: Fifo delivery fails sometimes
                  Clebert Suconic Master

                  Also, notice that the message will be on a client buffer if you are not consuming all the messages on a buffer.

                   

                  If you have a slow consumer, or if you are opening a consumer just to close it after received the message, you should set the consumerWindowSize=0 to disable client buffering. (that will have performance implications, as your throuhgput of messages will be maximized by your network lattency since you will need to send a packet from the client to receive a message from the server).