Esoteric functionality in JBoss Messaging

[genman]

I know even just the basics aren't there yet... But I want to make sure that the architecture might accomodate the following existing features:

1. Scheduled delivery, which is a useful feature for timer-style messages and retry delays for MDB.
2. Expiration of messages within queues (related to 1). (Incidentally, one interesting thing I found with the patch for JBossMQ was that allowing for specific NACKs allows the client to tell the server how to dispose of a message.)
3. Redelivery counts, mostly for reporting purposes and supporting a dead letter queue system for MDB.
4. This is not existing, but...I would allow users to map arbitrary headers for messages to specific column names. Assuming you go with Hibernate, this seems easily done.

Luckily, most of this stuff is handle-able just at the persistence layer, since we're dealing with non-consumed messages. The important thing is that certain columns are exposed. 1 and 2 would merely require a "fill" thread running, e.g. "select * from msgs where ttl > N or scheduled_date > N or foo = 'bar' or id > 1000" which would fill an in-memory series of heaps. You'd probably want separate in-memory heaps for ordinary messages to process, messages to expire, messages to enqueue and messages that are selector-optimized.

Depending on which heap a message is read from, it removes itself from the other heaps.

Shipping with some premade wrappers (for channels or whatever) that could simply consume a message (say, that failed 10 times or expired), and push it onto another queue atomically would be good. From an application design perspective, must often take an existing message and, based on some condition (such as a slow/unreliable customer), move it in a separate MDB pool. If this were configurable, it would be awesome.

[ovidiu.feodorov]

1, 3: Messaging will be backward compatible with JBossMQ, and since MQ offers retry delays and retry count configuration for MDBs, we must have them here as well. Compatibility refers to the format of the destination deployment descriptor, so "old" destinations can be deployed without any change, not to the internal implementation, of course.

2: A configurable way of disposing of undeliverable messages does not seem particularly difficult to implement. It requires support at core level, but the current architecture can be extended to allow it.

4. Could you please elaborate a bit on this one? I am thinking about a use case.

I created a JIRA tasks for it http://jira.jboss.org/jira/browse/JBMESSAGING-127 as a reminder, but we shouldn't use it for discussion until we define the specifications here. This thread is linked to it.

[genman]

Mostly I'd like to have better performance with using message selectors.

When dealing with a large queue (say, 10,000,000 rows), with a message selector, the only possible way to have decent performance is to write a query using the indexed row.

Hibernate might make it fairly easy to take a message selector and come up with an equivalent HQL query, which then becomes a SQL query. Depending on the Object-Relational mapping the user defines for the persitence manager, this might automatically take advantage of an indexed parameter.

Although this is more of a separate discussion, I'm assuming the new JMS is going to have lazy-loading of queues for the first pass? At the very least, I would hope that upon start-up only the references are loaded, and not the entire message.

[ovidiu.feodorov]

When dealing with a large queue (say, 10,000,000 rows), with a message selector, the only possible way to have decent performance is to write a query using the indexed row.

OK. Yes, this will be possible. It is not currently possible in alpha, where I am applying the filter after the rows were returned from the database but it is relatively easy to refactor and pass the responsibility to the database.

Although this is more of a separate discussion, I'm assuming the new JMS is going to have lazy-loading of queues for the first pass? At the very least, I would hope that upon start-up only the references are loaded, and not the entire message.

Yes. http://jira.jboss.org/jira/browse/JBMESSAGING-40

[genman]

I noticed my name come up regarding porting this "esoteric features."

I wish I had time (and a cookie) for working on these features. Unfortunately, I'm working at a new company that's adamently Weblogic at the moment and I might not be able to justify working on JBoss Messaging.

And unless you have a customer demand for scheduled messages and improved expiration features, there's no likely no reason to work on these features.

Although it seems relatively easy to implement, there's a couple of things that I see as complicating matters. One, you will probably want to add indexes for scheduled delivery time. And then your queries are additionally more complicated, not to mention additional issues with data migration. I think it would have been better if Hibernate or something was used for managing the queries as I expect changing the schema is a big hassle.

Anyway, if you're curious on my thoughts, let me know. I do like thinking about the project.

[ovidiu.feodorov]

Thanks. We need to get those done sooner rather than later. Will keep the discussion public.