1 2 Previous Next 17 Replies Latest reply on Jan 8, 2008 6:49 AM by assaf49

    Different Message id for each listener

    assaf49

      Hi,

      We have several listeners (each with a different selector) for the same Topic.
      The order of the messages per each listener (we need FIFO) is very important for us.

      We understand that after connection failure, the "failure recovery" mechanism does not guarantee ordering of messages.
      So we would like to identify "out of order" messages by tracking non-sequantial Message Ids.
      However this only makes sense if each listener (different selectors remember?) has it's own messageId.
      Is it possible to get a MessageId per listener?

      thanks
      Assaf

        • 1. Re: Different Message id for each listener
          timfox

          You shouldn't make any assumptions about what a message id contains apart from that it must start with 'ID:' and be unique per message. (This is what is mandated by the JMS spec).

          Making any such assumptions further than that will make your code highly brittle and non portable.

          Just because JBM ids currently *seem* to be made up of ID:JBM- plus some kind of sequence is no guarantee they actually are, or they will continue to do so in the future.

          If you want to maintain a counter you should add your own counter property in the message and maintain it yourself.

          • 2. Re: Different Message id for each listener
            assaf49

            Thanks.
            Ok i understand i should'nt make any assumptions regarding the message id.

            We can't add multiple counters, cause we have many JMS listeners with different (and OVERLAPPING) selectors.

            Is there any future plan to refine the JBM "failure recovery" mechanism so that message order (FIFO) is maintened even after network failures?
            If so in whch JBM version will this be available?

            • 3. Re: Different Message id for each listener
              timfox

              If there is only one consumer on a queue, JBM already maintains delivery order, even after failure.

              If there is more than one consumer on the queue this is not really possible. Since order delivery order in the queue will be determined by the order the consumers failed - i.e. non deterministic.

              • 4. Re: Different Message id for each listener
                assaf49

                In our case we use many listeners on one Topic (not a Queue).
                So i reckon (please correct me if i am wrong) it is possible/deterministic to maintain ordering for all listeners on the same Connection after failure.

                For example, after a Connection failure, make sure that "failed" messages are redelivered Before new messages (coming after failure).
                Am i making sense?

                • 5. Re: Different Message id for each listener
                  timfox

                  I'm assuming you are using durable subscriptions on your topic, or they won't survive failure anyway.

                  On that assumption, order after failover will be maintained.

                  • 6. Re: Different Message id for each listener
                    assaf49

                    thanks for your quick reponse :)

                    No, our Topic listeners are NOT durable,
                    we must maintain order of messages only whilst listeners ARE Connected.

                    I reckon that ordering (whilst Connected) is possible for such undurable listeners right?

                    This seems like an important feature.
                    Do you plan to support it (it will spare us a lot of trouble :)?

                    • 7. Re: Different Message id for each listener
                      timfox

                      I don't understand your use case.

                      If your subscriptions are non durable, then they won't survive failure, therefore saying order is maintained is meaningless.

                      Maybe you need to explain your scenario in more detail...

                      • 8. Re: Different Message id for each listener
                        assaf49

                        Scenario:
                        A JMS Connection is used to create 10 JMS listeners on Topic MyTopic.
                        Each Listener has a different selector (some have OVERLAPPING selector properties).


                        1) Producer sends Message A.
                        2) Some Connection failure occurs on the Listeners side
                        3) Producer sends Message B
                        4) Redelivery mechanism redelivers Message B (can this happen?)
                        5) Redelivery mechanism redelivers Message A (order is lost and the listener is not even aware of it)

                        • 9. Re: Different Message id for each listener
                          timfox

                           

                          "assaf49" wrote:
                          Scenario:
                          A JMS Connection is used to create 10 JMS listeners on Topic MyTopic.


                          By "listener" I'm assuming you mean non durable subscription (?)

                          And I'm assuming you're not using MDBs since you haven't mentioned them.


                          Each Listener has a different selector (some have OVERLAPPING selector properties).

                          1) Producer sends Message A.

                          2) Some Connection failure occurs on the Listeners side


                          Does message A get received before failure?

                          I will assume yes since you haven't specified.

                          What do you mean "connection failure"? Does the client process crash? I will assume yes here too.


                          3) Producer sends Message B




                          Does the producer send message B before or after the non durable subscription is recreated?

                          Please note that non durable subscriptions *do not survive* failure so any messages sent when they are non created (i.e. after the crash but before the recreation) will be *lost*.

                          This is perfectly normal behaviour, see JMS spec for more details about differences between non durable and durable subscriptions.

                          • 10. Re: Different Message id for each listener
                            timfox

                            So, to clarify:

                            If you are using straight JMS (no MDBs) and using non durable subscribers to subscribe to the topic, then if the connection between the client and the server is lost, any messages sent to the topic while the connection is down, will be "lost" (since the subscription is non durable).

                            When you recreate the non durable subscription after failure you will continue to receive messages sent after the connection is recreated.

                            For a durable subscription, since it is "durable", it continues to accumulate messages even while the connection from the client is down. When you reconnect you will continue to receive messages including those sent while the connection is down.

                            In *both* cases, order is always maintained as per JMS spec (i.e. order of messages sent from any particular session is maintained), although there may be gaps in the order in the case of the non durable subscriber.

                            • 11. Re: Different Message id for each listener
                              assaf49

                              Your assumptions where right:
                              1) We do NOT use MDBs
                              2) Our listeners are non durable topic listeners.

                              Scenario Clarified:
                              1) Producer sends Message A.
                              2) BEFORE Message A is recieved, some Connection failure occurs on the Listeners side. Note that the client does NOT crash, it's just a 'small' connection problem.
                              3) JBM is trying to recover the Listeners Connection failure.
                              4) Producer sends Message B (while JBM is still in process of trying to recover the Connection failure).
                              5) JBM manages to recover the Connection failure (as to your question, remember that there is no need to Recreate the listeners because they never crashed in the first place).
                              6) Redelivery mechanism redelivers Message B to Client (can this happen?)
                              7) Redelivery mechanism redelivers Message A to Client (order is lost and the Listener is not even aware of it)

                              • 12. Re: Different Message id for each listener
                                timfox

                                Hello Assaf, I am still having trouble understanding your scenario:

                                "assaf49" wrote:


                                Scenario Clarified:
                                1) Producer sends Message A.
                                2) BEFORE Message A is recieved, some Connection failure occurs on the Listeners side. Note that the client does NOT crash, it's just a 'small' connection problem.
                                3) JBM is trying to recover the Listeners Connection failure.


                                I'm not sure I understand what you mean by "JBM is trying to recover...".

                                Are you using a clustered connection? If so, JBM will attempt to failover to a new server if it detects the connection is lost.

                                Maybe you have coded your own exception listener and reconnect logic?


                                4) Producer sends Message B (while JBM is still in process of trying to recover the Connection failure).
                                5) JBM manages to recover the Connection failure (as to your question, remember that there is no need to Recreate the listeners because they never crashed in the first place).


                                Not sure what you mean by "JBM manages to recover..."


                                6) Redelivery mechanism redelivers Message B to Client (can this happen?)


                                No redelivery mechanism should kick in unless you have coded your own exception listener and have implemented some retry logic.


                                7) Redelivery mechanism redelivers Message A to Client (order is lost and the Listener is not even aware of it)



                                Order should never be lost.

                                • 13. Re: Different Message id for each listener
                                  assaf49

                                  Hi Tim, we are not using a clustered connection.
                                  However, we read in Jboss documentation (see http://docs.jboss.org/jbossas/javadoc/4.0.3SP1/j2ee/javax/jms/Connection.html)
                                  that "a JMS provider should attempt to resolve connection problems itself before it notifies the client of them".

                                  This probably means that JBM has some kind of automatic mechanism for resolving connection problems. Am i right?
                                  If so, do this mechanism ensure ordering of messages that are sent whilst JBM is resolving these connection problems?

                                  Another question : We have an Exception Listener that tries to recover a problematic connection. Can we be sure that messages recieved after our connection recovery are in order?

                                  • 14. Re: Different Message id for each listener
                                    timfox

                                     

                                    "assaf49" wrote:
                                    Hi Tim, we are not using a clustered connection.
                                    However, we read in Jboss documentation (see http://docs.jboss.org/jbossas/javadoc/4.0.3SP1/j2ee/javax/jms/Connection.html)
                                    that "a JMS provider should attempt to resolve connection problems itself before it notifies the client of them".

                                    This probably means that JBM has some kind of automatic mechanism for resolving connection problems. Am i right?
                                    If so, do this mechanism ensure ordering of messages that are sent whilst JBM is resolving these connection problems?

                                    Another question : We have an Exception Listener that tries to recover a problematic connection. Can we be sure that messages recieved after our connection recovery are in order?


                                    We always guarantee order as the JMS spec mandates.

                                    1 2 Previous Next