Storing only most recent message in queue

[slloyd]

We have a specific requirement in our application to only make use of the most recent message. We are streaming messages onto a queue at a rate of hundreds per second. The scenario that we want to achieve is essentially to be able to take the most recent (ie last-added) message from the queue, clear the queue (ie of any older messages, which we no longer require), then go off and process the message that we just pulled off the queue. While we are processing it, more messages appear on the queue. We then repeat the process.

Is something like this even possible with ActiveMQ? I noticed that JBoss' HornetQ has the concept of a "Last Value Header" (see this post), by which I can submit a message to a queue with a specific value for the HQLVQ_NAME header, and if any other message already exists on the queue for that header value, the newer message will overwrite the older one, such that there will only ever exist one (ie the most recent) message with that header value. This is exactly the behaviour we are looking for, but as we already have some time and effort invested in Camel and ActiveMQ, we would ideally prefer to stay with these if at all possible.

Thanks,

Shannon

[garytully]

Do you really need Queue semantics?

With a topic subscriber in ActiveMQ it is possible to configure a pendingMessageLimit strategy and eviction policy that will ensure that while a batch of messages are being processed, another batch of most recent messages will be stored pending dispatch to that subscriber. Messages that exceed that pending batch limit will be ignored.

With respect to the "Last Value" feature for a queue. This does not exist out of the box but could be easily implemented as a  broker interceptor .

For particular queues you could keep track of the last message id with a particular header value (by intercepting Broker.send()) and remove the predecessor before dispatching the most recent. All the apis to do this are easily accessible from an interceptor.

[slloyd]

Thanks for the pointer to the interceptor documentation - very useful. After a bit of digging around, it appears that I should be able to achieve what I am after by simply intercepting the send() call in the filter and using a JMS selector (using the header value) to remove any matching messages from the Queue. So long as the removeMatchingMessages call is not too slow, this should work nicely (with very little code).

Thanks.