13 Replies Latest reply on Feb 3, 2009 10:05 AM by François Duchatelet

    Configure a queue to be ordering aware

    Yong Hao Gao Master

      So far we have implemented the strict ordering through message producer. The messaging client can control the message ordering through creating ordering groups via the producer. This is one way to do it however. Another way is by configuring a specific Queue to be an ordering group Queue. Once thus configured, the queue destination becomes a ordering aware destination -- meaning that any messages that are targeted at this queue will naturally become ordered. This way will allow the users to get message ordering without coding.

      To accomplish this end, we can add a new property to QueueService. For example we name this property EnableOrderingGroup, if the value of this property is true, the Queue is ordering message group queue, as in

       <mbean code="org.jboss.jms.server.destination.QueueService"
       name="jboss.messaging.destination:service=Queue,name=strictOrderingQueue"
       xmbean-dd="xmdesc/Queue-xmbean.xml">
       <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
       <depends>jboss.messaging:service=PostOffice</depends>
       <attribute name="EnableOrderingGroup">true"</attribute>
       </mbean>
      


      or in clustered case

       <mbean code="org.jboss.jms.server.destination.QueueService"
       name="jboss.messaging.destination:service=Queue,name=strictOrderingDistributedQueue"
       xmbean-dd="xmdesc/Queue-xmbean.xml">
       <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
       <depends>jboss.messaging:service=PostOffice</depends>
       <attribute name="Clustered">true</attribute>
       <attribute name="EnableOrderingGroup">true"</attribute>
       </mbean>
      


      The order of each message is determined by the time of arrival at the target queue, with the earliest arrived message being the foremost of the delivering list.


      https://jira.jboss.org/jira/browse/JBMESSAGING-1416

        • 1. Re: Configure a queue to be ordering aware
          Tim Fox Master

          I think a better place to configure this is in the connection factory, not the queue.

          • 2. Re: Configure a queue to be ordering aware
            Yong Hao Gao Master

            we can do both ways.

            • 3. Re: Configure a queue to be ordering aware
              Tim Fox Master

              I would rather just have it configured in the connection factory, not the queue.

              This means the queue can be used for both ordered and non ordered messages. Also it's more consistent with message grouping in JBM 2.0

              • 4. Re: Configure a queue to be ordering aware
                Yong Hao Gao Master

                I agree with your reason. There is one exception, if user wants the ordering of messages to be determined by the time they arrive at the target queue (the time is the machine time that hosting the queue), the ordering has to be calculated on the server.

                Also if user wants strict ordering from multiple producers, the control has to be at the server side.

                • 5. Re: Configure a queue to be ordering aware
                  Tim Fox Master

                   

                  "gaohoward" wrote:
                  I agree with your reason. There is one exception, if user wants the ordering of messages to be determined by the time they arrive at the target queue (the time is the machine time that hosting the queue), the ordering has to be calculated on the server.


                  I don't understand that, ordering is always calculated on the server. Perhaps you could explain the point in more detail....

                  • 6. Re: Configure a queue to be ordering aware
                    Yong Hao Gao Master

                    sorry it's confusing. Calculate is not proper I think. Let me quote Taro's


                    Let me explain for your sure. Assume multiple producers, a queue and a consumer. Then, assume a scenario like:

                    Time
                    t1 Producer1 ---> Message1 ---> Queue1
                    t2 Producer2 ---> Message2 ---> Queue1
                    t3 Producer1 ---> Message3 ---> Queue1
                    t4 Producer2 ---> Message4 ---> Queue1

                    Where Time "tn" (n is integer) means the one when Queue1 receives a message from a producer
                    (the time relies on the clock on machine of Queue1).

                    Then, Queue1 has Message1, Message2, Message3 and Message4 in order and the consumer consumes the messages in this order.




                    • 7. Re: Configure a queue to be ordering aware
                      Tim Fox Master

                      Like i say, ordering is always done on the server anyway.

                      I'm not sure what you're point is here.

                      • 8. Re: Configure a queue to be ordering aware
                        Yong Hao Gao Master

                        Hi Tim, my point is how can you determine the ordering of message? I can think of two ways, one is determined by the senders, the sender can say message1 is the no1 and message2 is the no2, so JBM will obey this order. The other is determined by time, that is no matter who sends those messages, the order of them is determined by their time of arrival, but whose time should be used as the standard to judge the ordering?

                        • 9. Re: Configure a queue to be ordering aware
                          Tim Fox Master

                          We order when messages hit the server.

                          • 10. Re: Configure a queue to be ordering aware
                            Tim Fox Master

                            So.. I'm struggling a bit here to see where this thread is going.

                            Let me try and piece it together.

                            The way the messaging ordering is currently implemented in JBM is messages are ordered when they hit the server. This is the way it works/will work in JBM 2.0.

                            Ordering messages by adding a time stamp on the *client* is not something I have heard of before, and I do not know of any other messaging systems that implement it that way.

                            Using a timestamp from the client would have issues with clock synchronization on the client. The differences between timings on clients would probably far outweigh the time taken to send a message from client to server, thus rendering it pointless. For that reason, that option does not seem worth pursuing.

                            There seems to be another issue on this thread regarding how this functionality is configured - whether the queue is configured as "ordered", or whether this is done on the connection factory.

                            This second issue seems unrelated to whether we use timestamp from the client or order on the server.

                            As I mentioned in my previous post I would prefer to configure on the connection factory since this is more consistent with JBM 2.0 and also does not restrict the entire queue to be "strictly" ordered only.

                            • 11. Re: Configure a queue to be ordering aware
                              Yong Hao Gao Master

                              Hi Tim,

                              So far what I've done with message ordering in JBM 1.4 is client controlled ordering. That means the producer can define that messages that are sent by me have to be received in the strict order of me sending them. The producer does this not by timestamping the messages. Rather it gives each message (in each ordering group) a sequence number. The target queue will order the messages by their sequence numbers, not timestamps.

                              This doesn't cover the case where user wants messages (whereever they come from) to be ordered at the server (like a gathering point). In that case the 'hit time' make sense.

                              I agree we should be consistent between 1.4 and 2.0. But we also need to make sure this is what the users would like to have.

                              Another question I'm thinking is, How the time and order is determined when messages are being sent to a distributed queue? If there are two nodes (diff machines), messages can hit either of the nodes and each node (machine) has its own timing.

                              • 12. Re: Configure a queue to be ordering aware
                                Tim Fox Master

                                 

                                "gaohoward" wrote:
                                Hi Tim,

                                So far what I've done with message ordering in JBM 1.4 is client controlled ordering. That means the producer can define that messages that are sent by me have to be received in the strict order of me sending them. The producer does this not by timestamping the messages. Rather it gives each message (in each ordering group) a sequence number.


                                Ok, well that's not how we discussed implementing it before.

                                The idea was supposed to be that the client sets the ordering group header (it's just a constant string) and the server makes sure messages from that ordering group are delivered in strict order. There should be no sequence on the client side.

                                In any case, a sequence on the client side doesn't work, since you could have many different producers sending to the same ordering group (e.g. ordering group "europe_orders") and they still need to be processed in strict order, but clearly you can't share the same sequence here.

                                Looks like the design needs a rethink.

                                • 13. Re: Configure a queue to be ordering aware
                                  François Duchatelet Newbie

                                  Hi,

                                  WRT to where to specify the option to order message (actually, it is more general than this option), I support Howard in that it should be configurable on the Destination as well as the CF.

                                  Why on the Destination:
                                  - the option can be activated without any code change.
                                  - this can be seen as a Destination characteristic after all.
                                  - it wouldn't force the sending application to use different CF to send the messages.

                                  Why on the CF:
                                  - like Tim said, so the same destination can be used with and without ordering.

                                  Bottom line, it all depends on the usage scenario.

                                  Actually, I like the ability to "add" characteristics to messages using the objects through which they pass:
                                  - producer
                                  - CF
                                  - transport channel(s)
                                  - router(s)
                                  - destination

                                  By characteristics, I mean all the properties that a message can have, such as:
                                  - ordering group,
                                  - TTL,
                                  - size constraints,
                                  - priority,
                                  - actually any message property,
                                  - message filters,
                                  ...

                                  Cheers,
                                  Francois