7 Replies Latest reply on Aug 9, 2012 11:30 AM by vincentdhs

    Consumer-Max-Rate (again...)

    vincentdhs

      Hi there,

       

       

       

      i actually have to extend a webapplication consisting of a producer, a consumer and a queue which is shared with other producing applications,

       

       

      Due to some requirements, i have to regulate the rate the consumer consumes the message to 5 á second.

       

      Unfortunately the setting of consumerMaxRate isn't acting the way i expecting it to act.

      Instead of consuming the messages at the given rate the consumers-rate is capped to 5 a second at average (e.g. after 5 seconds, 25 message are consumed at once).

      This is pretty much what is already described here:

               https://community.jboss.org/message/547680#547680 

       

       

      Are there any possibilities to provide this behaviour? Maybe a custom connection-factory or something like that?

      (I do not want to obfuscate the consumers source with code the jms-provider is allready responsible for).

       

       

      Any suggestions?

       

      Regards

       

      florian

        • 1. Re: Consumer-Max-Rate (again...)
          clebert.suconic

          "5 a second at average (e.g. after 5 seconds, 25 message are consumed at once)."

           

           

          If that's true it sounds like a bug.

           

          are you sure about this?

          • 2. Re: Consumer-Max-Rate (again...)
            vincentdhs

            I'm not sure if this is a bug.

             

            The implementing class is named: TokenBucketLimiterImpl and act like a TokenBucketLimiter (limiting the AVERAGE rate)

             

             

            If so, the features-list is misleading regarding the max-rate limitations.

            It says the rate is limited (and not the average..):

             

             

                      Also, on a per consumer basic you can specify the maximum rate, in messages per second, that a consumer is allowed to consume messages.

             

                       E.g. you could specify specify 1000 msgs/sec and the consumer will never consume messages faster than that.

            • 3. Re: Consumer-Max-Rate (again...)
              clebert.suconic

              I tried to replicate what you were saying with TokenBucket directly and I couldn't see an issue.

               

              Are you sure you are setting up this correctly?

              • 4. Re: Consumer-Max-Rate (again...)
                vincentdhs

                I tried a minimal example using the following runtime configuration:

                 

                - JBoss 6.0 

                - HornetQ 2.1.2

                 

                And consiting of an MDB:

                 

                package com.regiocom.bpo;

                 

                import javax.ejb.ActivationConfigProperty;

                import javax.ejb.MessageDriven;

                import javax.jms.JMSException;

                import javax.jms.Message;

                import javax.jms.MessageListener;

                import javax.jms.TextMessage;

                 

                 

                @MessageDriven(activationConfig={

                                    @ActivationConfigProperty(propertyName="destinationType",propertyValue="javax.jms.Queue"),

                                    @ActivationConfigProperty(propertyName="destination",propertyValue="java:/queue/ServiceQueue"),

                                    @ActivationConfigProperty(propertyName="maxSession", propertyValue="1")

                                })

                public class JMSConsumer implements MessageListener{

                 

                    public void onMessage(Message message) {

                        try {

                            System.out.println("Message read:"+((TextMessage)message).getText());

                        } catch (JMSException e) {

                            e.printStackTrace();

                        }

                    }

                }

                 

                 

                 

                 

                Using a resourceadapter, where i added the following to the ra.xml

                 

                <config-property>

                        <description>The consumer max rate</description>

                        <config-property-name>ConsumerMaxRate</config-property-name>

                        <config-property-type>java.lang.Integer</config-property-type>

                        <config-property-value>1</config-property-value>

                      </config-property>

                 

                 

                The used queue is filled by an webservice, which is adding 10 messages at once.


                When first started, each messages timestamp (jboss-log) differs exactly by one second.

                 

                 

                Now, if i am wait for 5 seconds, the first 5 messages are consumed at once and only the remaining 5 messages are consumed in steps of one second.

                 

                 

                So actually, the consumer consumes 5 msgs/sec while 1 msg/sec is configured.

                • 5. Re: Consumer-Max-Rate (again...)
                  vincentdhs

                  Problem solved by using Springs @Scheduled-Annotation.

                   

                  Implemented a Class with a scheduled method, which received (NO_WAIT) the messages from the queue.

                  1 of 1 people found this helpful
                  • 6. Re: Consumer-Max-Rate (again...)
                    chrisabaird

                    I've run into this issue as well.  Looking at the source (2.2.14) it appears that the rate limiting is done using the "token bucket" method (as referenced in the code http://en.wikipedia.org/wiki/Token_bucket).  I believe I'm looking for a version of leaky bucket (http://en.wikipedia.org/wiki/Leaky_bucket#The_Leaky_Bucket_Algorithm_as_a_Queue).  This would be a useful (for me anyway :-) ) addition.  If that is not in scope, perhaps the ability to set the implementing rate limiting class.

                    • 7. Re: Consumer-Max-Rate (again...)
                      vincentdhs

                      If someone from HornetQ would provide me a spec on the behaviour of the integration (e.g. which file should be used for configuration)

                      i would love to integrate it.