-
1. Re: Limiting the consumers & producers of a queue
It seems like your real requirement here is to ensure that all messages from a queue are processed by only one consumer.
Limiting consumers is one solution to that problem, another more flexible way is message grouping - which guarantees that messages in a specific group always get processed by the same consumer.
http://jira.jboss.org/jira/browse/JBMESSAGING-375
Message grouping is scheduled for JBM 2.0, There are no plans at present to put this in 1.4.x. -
2. Re: Limiting the consumers & producers of a queue
Yes, that is my real requirement. I have a question about this grouping feature. If two clients connect with the same groupId, then what happens? In scenarios referenced in that Websphere blog post I linked, that's what would happen in some edge-case scenarios. Hopefully your grouping feature would cancel the previous subscriber using that groupid -- in order for this to be a solution to this problem.
-
3. Re: Limiting the consumers & producers of a queue
The way message grouping would work is as follows:
Before you send a message you set a special header on the message - not sure what it is going to be called yet, but let's call it GROUP_ID for now.
Before JBM delivers the message to a consumer, it checks if the GROUP_ID header is present, if it is then it looks up to see if it has a consumer registered for that group id, if it has then it sends the message to that consumer, if it hasn't then it chooses a consumer, registers that consumer against that group id then sends the message to the consumer.
So you can see that all messages with a specific value of GROUP_ID will be consumed by the same consumer (even though there may be multiple consumers on the same queue).
GROUP_ID is not specified by the clients that connect.
This is more flexible than limiting consumers since you can have multiple consumers consuming off the same queue with different values of GROUP_ID.
E.g. consumer1 deals with all the GROUP_ID=REDHAT_STOCK price messages, and consumer 1 deals with all the GROUP_ID=IBM_STOCK price messages. -
4. Re: Limiting the consumers & producers of a queue
If group_id is not specified by the clients that connect then where is it specified? (i.e. how do I set REDHAT_STOCK in your example)
I tend to think my suggested implementation is more straightforward to address my use-case but perhaps you have other use-cases in mind for this feature. -
5. Re: Limiting the consumers & producers of a queue
"dsmiley" wrote:
If group_id is not specified by the clients that connect then where is it specified? (i.e. how do I set REDHAT_STOCK in your example)
Sorry, to clarify, GROUP_ID is specified by the sender (consumer id isn't)
I tend to think my suggested implementation is more straightforward to address my use-case but perhaps you have other use-cases in mind for this feature.
Yes, the other examples I have in mind are the ones where you can have multiple consumers on the same queue with different values of GROUP_ID. -
6. Re: Limiting the consumers & producers of a queue
Everyone: we talked about this on IRC and I'll elaborate on this for everyone's benefit:
Once a particular consumer of the queue gets a message with a particular group_id (this is initernally managed, the consumer doesn't know anything about this stuff), it'll be the sole recipient of messages with that group_id as long as it's still connected. So in my scenario, the producer is always going to use the same group_id. This should handle the edge-cases I referenced. It'd be nice to have a feature to kick-off a consumer that you don't want consuming any more but that's not possible. -
7. Re: Limiting the consumers & producers of a queue
Maybe it's a little bit late to join the conversation, but better late than never. :)
The thing is that the message group idea only solves the problem if we want to limit the consumers to only one (1). If we want to have, let's say, max 3 consumers, then the message group idea is no longer useful.
For example: Let say that some operation or process needs (or occupies) 1 core in the processor to complete. Let say that we have a quad core processor (4 cores) in our server. We want to save one core for other operations. That leaves us to 3 cores to run 3 simultaneous processes. In this scenario it would be great to limit the consumers to 3. Maybe through an attribute in the settings-file for the specific queue and/or even set the limitation programatically.
Of course, we could create a singleton pool with mutex- or semafore-technology in each consumer to control the number of simultaneous processes. But this would (could) lead to transaction timeouts if the wait is too long between the first and last loaded consumer, which could fail the whole idea with QoS.
timfox: In your blog, you have described nicely the consumer flow control between the queue and the consumer:
http://jbossfox.blogspot.com/2008/03/mysteries-of-flow-control-explained.html
Is there a way, in the configuration or JMX bean, to control the number of tokens in the consumer flow control? If there are, I would really like to know how to manipulate them. That would really solve my problem. If not, is there a plan to make them visible in future versions of JBoss Messaging?