4 Replies Latest reply on May 16, 2012 11:24 AM by ataylor

    Message reception order by topic consumers across sessions

    btrip

      We are trying to understand in which messages in a HornetQ topic are consumed by clients. More specifically, we have the following scenario:

      - A single producer is spawned each minute by a quartz job, generates a set of messages and sends them to a topic, ordered in a specific manner (database timestamp)

      - Multiple consumers are subscribed to the topic

       

      Questions:

      a) Will the consumers always receive the messages from the topic in the same order as the producer put them in the topic?

      b) If not, is there any way to guarantee that the ordering will be correct, by other means (e.g. by replacing topics with queues, one per consumer?). We are only interested in the order in which each consumer receives the messages, not the overall order amongst consumers. However, we are interested in the order of message reception extending beyond the producer lifetime, i.e. if message A is sent the first time the producer is spawned and message B is sent the second time the producer is spawned, then message A should be received by each consumer before it receives message B.

       

      We have not seen a clear enough answer on this question, although there are hints in the JMS spec that the ordering of the messages is only guaranteed in a single session, which in our example is not the case (the producer will create a new session every time it is spawned).

      Also, in the FAQ section, we have found the following sentence:

      "HornetQ will always guarantee order of delivery at the producer level if you use a single consumer per queue without selectors.". Does this mean that the order of delivery to consumers is only guaranteed to be the same as the order the messages were added to the queue within a single JMS session, and messages added in different sessions may arrive in different order than the order in which they were added to the queue?

      We are very interested in performance, so the option of disabling buffering on the consumer is not acceptable. Furthermore, the solution of message groups cannot be applied to our requirements due to specific business requirements.

       

      Many thanks,

      Babis

        • 1. Re: Message reception order by topic consumers across sessions
          clebert.suconic

          Unless you redeliver.. the messages are consumed at the same order of the producer.

           

           

          Now, if you do something like this:

           

          producer1 (thread 1)

          msg1_1

          msg1_2

           

           

          producer2 (thread 2)

          msg_2_1

          msg_2_2

           

           

          we gurantee that msg_2_2 will be delivered after msg_2_1, and msg1_2 will be delivered after msg1_1

           

          if you need to stick the order of the producers together (I,e, msg1_1 and msg1_2 to stick together no matter what), then you need to take a look at message-grouping which will place a few more boundaries on these deliveries.

          • 2. Re: Message reception order by topic consumers across sessions
            btrip

            Hello Clebert,

            thank you for your answer.

             

            Let me further clarify our case.

            We don't have parallel threads/concurrent producers. Each producer runs sequentially, some time after the previous one has completed its job and is terminated. Thus, at any given time, only one producer instance is alive and sending messages to the topic. The producer code called to send messages is always the same, we refer to different producers only because we have a new JMS session.

             

            More specifically, based on your example, we assure that thread2 runs after the completion of thread1. In this case, we would like to know if it is guaranteed that msg_2_1 will be consumed in a timely order, i.e. after 1_2 and before 2_2.

             

            Our scenario, in more detail,  is the following:

             

            1) Ordered sending of messages to the topic:

            Producer P1 sends two messages to the topic, in the following order:

            1st message sent: msg1_1, second message sent: msg1_2

            P1 dies (session is terminated).

             

            We assume that the topic at this moment contains:  msg1_2, msg1_1 and any consumer that would receive messages, would first receive msg1_1 and then msg1_2.

             

            After a while, producer P2 is spawned.

            P2 sends another two messages to the same topic, in the following order:

            1st message sent: msg2_1, second message sent: msg2_2

            P2 dies

             

            2) Multiple Consumers have been subscribed to the Topic. Assuming that a consumer C1 starts receiving messages after P2 has died, what will the order of messages C1 will consume be?

             

            Is it guaranteed that the order of messages received by C1 will be as follows?

             

            1st message received: msg1_1

            2nd message received: msg1_2

            3rd message received: msg2_1

            4th message received: msg2_2

             

            Many thanks again,

            Babis

            • 3. Re: Message reception order by topic consumers across sessions
              btrip

              Any news on this?

              • 4. Re: Message reception order by topic consumers across sessions
                ataylor

                like clebert says ordering is gauranteed between a single producer and consumer(s). Theres nothing in the spec that defines ordering between multiple producers and consumers, however, if you can gaurantee that the first producer has sent and had acknowledged its messages before the second then you can assume the order holds.