AMQ-1112: Removing expired messages

[selikoff]

Has the Fuse team considering incorporating AMQ-1112 into future releases of the Fuse message broker?  As this Blog Article points out, if you are using message selectors, such a fix is vital to prevent hung queues.

The root of the issue is that the message broker does not remove messages as soon as they expire, but rather the next time a message is popped off the queue.  The AMQ-1112 fix inserts a worker thread that checks X number of messages of a queue every Y milliseconds (with X and Y being configurable parameters) and expires messages found to be out of date.

I've reproduced the 'hung queue' issue as mentioned in the blog.  Simply stated, create a queue with size = 10, then create 10 messages for a specific message selector, say myWidget=6, and send them to the queue.  Now create 10 more messages for a different message selector, say myWidget=7.  At this point, if a listener with (myWidget=6) does not connect to the queue or is otherwise unavailable, no widget (myWidget=7 or otherwise) will be able to obtain any messages even though some were sent and the queue is otherwise hung.  Only two listeners will be able to clear the queue, those with no message selector set (in this case they get all the messages) and those set to myWidget=6, which may be offline.

I reproduced the issue with both small and large queue sizes and it does lock up the queue.  Any idea if and when AMQ-1112 or a similar solution may be incorporated into Fuse?

[garytully]

The fix for AMQ-1112 will be in the next FUSE Message Broker release that is tracking the ActiveMQ 5.3.1 branch at Apache.

To validate that it solves your use case, you can try out a current snapshot of the next release from:

http://repo.fusesource.com/maven2-snapshot/org/apache/activemq/apache-activemq/5.3.0-psc-01-SNAPSHOT/

or use the 5.3.0-psc-01-SNAPSHOT maven tag and the fusesource repository.

At the moment, we don't have a hard release date.

Edited by: gtully on Jan 7, 2010 1:44 PM

[selikoff]

Hi Garry,

I actually merged forward your changes for AMQ-1112 into a local copy of the Fuse source and can confirm, this does indeed solve the issue.  Not only does this prevent 'hung' queues for people using message selectors but memory usage has been significantly reduced.  Given the number of files I had to change to incorporate AMQ-1112, I'd feel safer with an official/verified release.

By the way, thanks for working on this issue.

Cheers,

-Scott