Client Throttling when the messages are published to Queue with long delays

[mail.jeeva]

Here is a scenario

     Publish 1000 messages initially to a queue, at once (using a for loop)

     Consume the messages using Message Listener with a thottling of 1 message per second

     [The messages are consumed as expected @ 1 per sec]

     After all the 1000 messages are consumed, there is a delay in publishing messages to the queue, say an hour or so.

     Again publish 1000 messages to the same queue, at once (using a for loop)

     Now I see all the messages are getting consumed immediately, without the throttling effect.

     Any idea if the "TokenBucketLimiterImpl" logic takes acre of such delays?

     Can any one guide me how to address this issue?

Thanks,

Jeevaraj

[ataylor]

The token bucket algorithm works in real time not a fixed amout of time, so 60 messages per second will arrive in the first 60 seconds either one every second if u start delivering straight away or all 60 together if u wait 60 seconds.

You could solve your problem by using scheduled delivery when u send a message, i.e., keep a counter of what message u are sending and add a second each message, something like

time = current time

schedule message time +1 secs

schedule message time + 2 secs

etc, etc.

There is a schedule message example in the distro

[mail.jeeva]

Thanks Andy.

I believe the producer rate limiting also wont help as same token bucket algorithm is used there as well.

[timfox]

I don't really understand what the problem is here.

You have told the consumer to consume a max of 1 msgs / sec and that's exactly what it has done.

[bclevelandsview]

I think this is a pretty common use case.  The idea is to use a queue to throttle the load onto a backend resource that has a finite throughput per second.  I'm using HornetQ to route and manage traffic to SMS providers.  One of these requires a strict limit of 10 requests a second, and we get severe fees assessed for exceeding this rate.

I have other methods for dealing with this, but it would be much easier if I could specify that a queue will never exceed a specific message rate per second.

[timfox]

Bob Cleveland wrote:

I have other methods for dealing with this, but it would be much easier if I could specify that a queue will never exceed a specific message rate per second.

You can specify that a consumer does not exceed a certain rate, is that not what you want?

[mail.jeeva]

Yes, the consumer does not exceed the specified throttle rate if the messages are published continuously without any delays (greater than the throttle rate) between them. But if there is a delay, all the messages received after the delay are consumed immediately.

For example, if the rate is set as 1 message per sec, and the messages are produced continuously (the delays between successive messages are less than 1 second) the consumer consumes at the specified rate.

Now let us assume there is a pause in message production.

All the previously produced messages are consumed at the same specified consumer throttling rate.

Again let us assume, a pause of an hour more, the message production resumes.

This time all the messages are consumed immediately. This, in effect, doesn't honor the specified consumer throttling rate. The issue is apparently with the token bucket limiter algorithm.

Following options may be considered for overcoming this issue:

1. Use a scheduled message production as specified by Andy Taylor, earlier in the thread in combination with the consumer throttling rate

2. Improvise the "TokenBucketLimiterImpl" by adjusting the last message time to current time - (1 / the specified throttling rate) 0 this needs a code change, so that the throttling rate is honored at all scenarios

3. Open up an interface for custom rate limiting algorithms

Thanks,

Jeevaraj

[clebert.suconic]

There is the maxConsume rate also.

You could limit the rate at producing.. and the rate at consuming.

[timfox]

Jeevaraj Dhanaraj wrote:

Yes, the consumer does not exceed the specified throttle rate if the messages are published continuously without any delays (greater than the throttle rate) between them. But if there is a delay, all the messages received after the delay are consumed immediately.

For example, if the rate is set as 1 message per sec, and the messages are produced continuously (the delays between successive messages are less than 1 second) the consumer consumes at the specified rate.

Now let us assume there is a pause in message production.

All the previously produced messages are consumed at the same specified consumer throttling rate.

Again let us assume, a pause of an hour more, the message production resumes.

This time all the messages are consumed immediately. This, in effect, doesn't honor the specified consumer throttling rate. The issue is apparently with the token bucket limiter algorithm.

I disagree with you. The algorithm does honor the specified consumer throttling rate.

You set the rate to 1 msg / sec. Over a period of an hour, you should be allowed to consume 3600 msgs. After 1 hour you have only consumed 1000, therefore you should be allowed to consume a further 2600. You only send 1000 further messags so these are immediately consumed.

The overall consumption rate is less than 1 msg/sec.

[timfox]

You should bear in mind, that the maximum rate you specify is the overall message consumption rate in msgs/sec since the consumer was created.

If you want some other behaviour, if you can define it precisely and it makes sense we can attempt to implement it, but as yet I haven't heard any tight definition of your requirement.

[mail.jeeva]

Hi Tim,

Thanks for taking time to help with my issue.

My requirement is that, irrespective of when the consumer was created, the consumption rate has to be limited based on the configured max-rate for any "one second window".

That means, if I configure 1 message per second as my max consumption rate, at any one second interval, I want only one message to be consumed, irrespective of when the consumer was created. This would ensure the system doesn't choke even if there was a pause in message production.

Thanks,

Jeevaraj