9 Replies Latest reply on Apr 19, 2010 4:22 AM by Tim Fox

    Push vs Pull cluster distribution

    Boris Lublinsky Novice

      It seems like in the current implementation of cluster distribution producer does distribution immedietly when a message is created. This means that a message is send to the consumer regardless of the speed that they can consume it with.

      It also means that once a message was delivered to consumer, even when additional consumer is added, the messages will not be redistributed.

      Am I missing something again?

      Is there a way to do distribution only on demand? When a consumer is ready to process it is pulling a message from producer.

        • 1. Re: Push vs Pull cluster distribution
          Tim Fox Master

          Consumer flow control (see user manual) prevents consumers from being overloaded with messages. (see user manual)

           

          To be honest though, I don't really think I understood your question properly.

          • 2. Re: Push vs Pull cluster distribution
            Boris Lublinsky Novice

            What I ment is allowing clusters to work as a queue manager.

            In the queue manager, you store your messages locally and allow clients to poll them (get the message when the client is ready), right?

            In the case of cluster, it seems like you are doing things differently - you push messages to the other nodes, regardless of how fast their clients are going to process.

            Why not switch. Message is always stored at the producer manager. When a consumer on any node of the cluster is trying to read a message, then the actual distribution happens. Pull vs Push

            • 3. Re: Push vs Pull cluster distribution
              Tim Fox Master

              Boris Lublinsky wrote:

               

              What I ment is allowing clusters to work as a queue manager.

              In the queue manager, you store your messages locally and allow clients to poll them (get the message when the client is ready), right?

              In the case of cluster, it seems like you are doing things differently - you push messages to the other nodes, regardless of how fast their clients are going to process.

              Why not switch. Message is always stored at the producer manager. When a consumer on any node of the cluster is trying to read a message, then the actual distribution happens. Pull vs Push

              That's basicallly the way JBoss Messaging used to work. However we changed distribution to always be round robin since the JBM (pull) method was just causing so much confusion from users - it was the source of a lot of support cases.

               

              This was discussed in another thread last week IIRC.

               

              The moral of the story is you can't please all the people all the time.

              • 4. Re: Push vs Pull cluster distribution
                Boris Lublinsky Novice

                Understood, but..,

                Is it possible to use Consumer flow control with cluster message distribution?

                It seems like a good solution to be able to control cluster distribution, but I can't figure out how to configure it. Any help with this? 

                • 5. Re: Push vs Pull cluster distribution
                  Tim Fox Master

                  Consumer flow control is already configured by default. See the flow control chapter in the user manual for more info.

                  • 6. Re: Push vs Pull cluster distribution
                    Boris Lublinsky Novice

                    I went through the manual.

                    My problem is that in the case of cluster you are creating connection dynamically and I do not see how to specify parameters in this case

                    • 7. Re: Push vs Pull cluster distribution
                      Tim Fox Master

                      You lost me again.

                       

                      Not sure what you mean by "My problem is that in the case of cluster you are creating connection dynamically and I do not see how to specify parameters in this case" and how this is related to consumer flow control.

                       

                      To be honest I am pretty confused by this whole thread. I am trying to answer to the best of my abilities but I'm finding it hard to understand where you're coming from.

                      • 8. Re: Push vs Pull cluster distribution
                        Boris Lublinsky Novice

                        I am trying to implement situation described on page 67 of a user manual, but for cluster connection. An example that is described there - slow consumers, is exactly the case that I have, but for a clustered architecture - my consumers are clustered queue managers. I need help to figure out how to configure clustered nodes to reach the same resulr

                        • 9. Re: Push vs Pull cluster distribution
                          Tim Fox Master

                          Slow consumers (setting consumer-window-size to zero) just means that each consumer will not buffer any messages from a queue. This allows for deterministic round robining between multiple consumers on the same queue.

                           

                          Consumers simply connect to a local queue, they have no knowledge of any other nodes in the cluster.

                           

                          With a cluster connection - this is not a consumer connection. Each cluster connection is composed of a core bridge which forwards messages. This is currently done in a round robin fashion too.

                           

                          Like I mentioned before, there is no "pull" distribution of messages between nodes like there was in JBoss Messaging. It is strictly "push" round robin.

                           

                          Hope that answers your question