12 Replies Latest reply on Sep 15, 2011 4:38 PM by bbansal

    Clarification on Durable messages for Topics

    billy.sjoberg

      Hi guys,

       

      Going over the HornetQ manual I fail to understand the section regarding durable subscribers.

       

      "

      11.1.18. Durable Subscription

       

      The durable-subscription example shows you how to use a durable subscription with HornetQ. Durable subscriptions are a standard part of JMS, please consult the JMS 1.1 specification for full details.

       

      Unlike non-durable subscriptions, the key function of durable subscriptions is that the messages contained in them persist longer than the lifetime of the subscriber - i.e. they will accumulate messages sent to the topic even if there is no active subscriber on them. They will also survive server restarts or crashes. Note that for the messages to be persisted, the messages sent to them must be marked as durable messages.

      "

       

      I have set up a test where I have a publisher A (actually a JBoss ESB flow) sending messages to a topic. Another flow (B) uses a durable subscription against that topic. The JMX console now shows that I have one Durable subscription against this topic.

       

      Once I send a message the JMX console tells me that I have one NonDurableMessageCount and zero DurableMessageCount's.

      I performed a test where:

       

      - I undeployed the subscriber B

      - Sent a message to the Topic.

      - Once the message was sent I restarted my AS.

      - Deployed subscriber B

       

      In contrary to the documentation the subscriber received the message even though it clearly was a NonDurable message.

      Can I guarantee that subscriber B will receive messages even though he was out when the messages are NonDurable?

       

      If not, does anyone have experience with JBoss ESB and how to send durable messages?

        • 1. Re: Clarification on Durable messages for Topics
          ataylor

          If the message is non durable then, after a server restart, the message will not be available, how are you setting the durablity of the message

          • 2. Re: Clarification on Durable messages for Topics
            billy.sjoberg

            The JMX values I saw was the result of not enough coffee.. It was actually my JMS browsing app (HermesJMS) which was registered as a non-durable subscriber on the topic in question.

             

            When you're mentioning setting the durability, how is that set on the message? I've set the DeliveryMode to persistent (message.setJMSDeliveryMode(DeliveryMode.PERSISTENT)), is that the same property used for both queue and topics? Or is Durable another property intended only for topics?

            • 3. Re: Clarification on Durable messages for Topics
              ataylor

              you need to set it on the producer not the message

              • 4. Re: Clarification on Durable messages for Topics
                bbansal

                hey Folks,

                 

                any idea about how to have durable topics with Stomp clients eg perl

                 

                My use case is I want to have 1 publisher and 5 different apps subscribing messages, the apps can go down and can come back online and should not miss messages from the topic.

                 

                1) I dont want to send messages to 5 different queues from the producer

                2) If I use a topic

                      -- If consumer goes down and came back it misses all the events in the down time.

                3) I am setting persistent => 'true' (also tried persistent => 1 ) in $client->send() call

                4) I dont see any attribute like queue.durable which I can set for a topic also

                • 5. Re: Clarification on Durable messages for Topics
                  ataylor

                  Ther first thing to mention is that topics are not durable, its the subscriptions that are durable. Unfortunately the Stomp protocol does not support this, however the good news is that HornetQ allows a way of doing this via headers.

                   

                  Take a look at the Stomp Test we have here particularly the durable subscriber tests

                  • 6. Re: Clarification on Durable messages for Topics
                    bbansal

                    Thanks Andy,

                     

                    I am able to configure durable subscribers for topics. (i.e. data is not lost and replayed back if the consumer goes down and comes back up etc.) I also need to support a durable topic consumer load-balanced across multiple machines.

                     

                    Q1 Is it supported right now ? if not is that a big issue or just work need to be done ?

                     

                    I am looking at StompSession.java : 192 - 210

                                QueueQueryResult query = session.executeQueueQuery(queue);

                                if (!query.isExists())

                                {

                                   session.createQueue(SimpleString.toSimpleString(destination), queue, null, false, true);

                                }

                                else

                                {

                                   // Already exists

                                   if (query.getConsumerCount() > 0)

                                   {

                                      throw new IllegalStateException("Cannot create a subscriber on the durable subscription since it already has a subscriber: " + queue);

                                   }

                                }

                     

                    I think if we improve handling of else case when the queue already exists and add more consumers here instead of the error, that should take care of it ? thoughts

                    • 7. Re: Clarification on Durable messages for Topics
                      clebert.suconic

                      We are only implementing Stomp 1.0 at the moment, in which we are implementing the newest 1.1 as we speak.

                       

                      1.0 doesn't include pings / pongs.

                       

                      We however controls pings / pongs...  so a client should be disconnected if not talking back to the server on a given time.

                       

                       

                      Having said that, you shouldn't have more than one consumer active at one point. You would need to wait the TTL for the consumer to be kicked out.

                      • 8. Re: Clarification on Durable messages for Topics
                        clebert.suconic

                        BTW: this is a limitation of the concept of durable subscriptions. You can't have more than one consumer on an active subscription. (Being JMS like).

                         

                         

                        Unless you find me a good reason to remove that constraint on stomp.

                        • 9. Re: Clarification on Durable messages for Topics
                          bbansal

                          Hey clebert,

                           

                          thanks for your comments, I think it make sense in terms of being compatible with JMS or stomp 1.0.

                          But from a practical perspective I think a common use case is one service publishing events and multiple services listening to them

                          and for scalibility perspective you need to scale out consumers as load-balnanced durable subscribers. Lack of this feature is big issue

                          in my mind.

                           

                          I think ActiveMQ supports this by defining composite destinations etc., what is the best alternative in hornetQ for that ? I am happy with core-api if we need we should add stomp support for it etc.

                           

                          Best

                          Bhupesh

                          • 10. Re: Clarification on Durable messages for Topics
                            clebert.suconic

                            We offer topic hierarchies on HornetQ. Have you looked at that?

                            • 11. Re: Clarification on Durable messages for Topics
                              clebert.suconic

                              I'm just wondering if we can't just remove that constraint. I don't see a good reason on Stomp:

                               

                               

                              I see a good reason on JMS since that's a JMS requirement. But AFAIK Stomp is not JMS.

                               

                              Anyone objects?

                              • 12. Re: Clarification on Durable messages for Topics
                                bbansal

                                +1,

                                 

                                I support adding a special flag in stomp to enable load-balanced durable susbscribers. I think it will solve a lot of use-cases.

                                 

                                Best

                                Bhupesh