Round-robin queue mode

[ulf.schroeter]

Is there a possibility to configure round-robin queues instead of standard fifo queues, were inserts of a new message exceeding a user- defined message count limit would simply 'overwrite' the oldest stored message in this queue ? Based on the documentation I think this is not possible except for the available special last-message queue case.

Having such a capability would be very helpfull to limit queue memory size especially for durable subscribers which are only temporarilly connected. Normal fifo queues can only be limited by dropping newest messages instead of oldest, last-message queus can only provide the single newest message, but not a configurabe count > 1 newest messages.

Would it be possible to implement such feature based on current Hornetq architecture ? Any comment highly appreciated.

[ataylor]

could you exlain a bit more what you mean by a 'round robin queue'. what is being is being round robined? if you replace the first message on the queue where does the next message go?

[ulf.schroeter]

Hi Andy,

maybe "last-x-messages queue" is a simpler explanation (were x specifies the user-definable upper count of stored messages). While a last-message queue only stores 1 last message, we would like to store the last x send messages. On new message insert - in case there are already x messages stored in this queue - the oldest stored message should be simply dropped from the queue.

Remark: I know that last-message queue operates on a message-property-level within a queue, but at the moment I am uncertain if we would need the discussed feature on message-property level or just on queue level).

To further illustrate our use-case: we would like to have a topic with multiple durable subscriptions (address + multiple bound persistent queues). Messages will be inserted continously e.g. 100 msg/min while the individual consumers will only re-connected at certain times e.g. once per hour for just 1 minute.

On re-connect each individual consumer should only receive the newest published 100 messages (= 1min) before re-connect and all follow-up messages send while connected/online for 1min, but not the 5800 messages (58 min) send previously after the consumer went offline in the previous activity cycle.

Hope this explains the targeted functionality better.

[ataylor]

maybe "last-x-messages queue" is a simpler explanation (were x specifies the user-definable upper count of stored messages). While a last-message queue only stores 1 last message, we would like to store the last x send messages. On new message insert - in case there are already x messages stored in this queue - the oldest stored message should be simply dropped from the queue.

Ok, I get it, the round robin comment thru me. So, no we currently cant do that. Implementing this wouldn't be too hard so feel free to raise a JIRA if you want. Unfortunately its unlikely that we could get round to doing this any time soon but we would always welcome contributions.

Remark: I know that last-message queue operates on a message-property-level within a queue, but at the moment I am uncertain if we would need the discussed feature on message-property level or just on queue level).

i would say it would be a new type of queue, maybe 'bounded queue' or something.

To further illustrate our use-case: we would like to have a topic with multiple durable subscriptions (address + multiple bound persistent queues). Messages will be inserted continously e.g. 100 msg/min while the individual consumers will only re-connected at certain times e.g. once per hour for just 1 minute.

On re-connect each individual consumer should only receive the newest published 100 messages (= 1min) before re-connect and all follow-up messages send while connected/online for 1min, but not the 5800 messages (58 min) send previously after the consumer went offline in the previous activity cycle.

One thing we would need to decide would be how to handle message buffering, the client could have 200 messages buffered, if it consumes 100 the 100 will be put back on the queue (usually the head), but this will break semantics as other new mesages may have arrived at the queue since. of course you could set the buffer size to 0.