REST PushConsumer Pool

[amdonov]

I'm using HornetQ's REST API and registering a single service as a push consumer with my queue. While I want all messages processed by a single service, I would like to process multiple messages concurrently. The only way that I've been able to achieve this result so far is by registering the same endpoint as a push consumer multiple times (currently 5, but it might go up). While this works, I would much rather have a dynamic pool of consumers that grows and shrinks according to load. Is there a way to configure this now? Are there any plans for this functionality in the future? If not, I would appreciate any pointers on how I might implement it myself. I've looked at PushConsumer.java briefly, but I'm not very familiar with the HornetQ codebase.

Thanks.

[clebert.suconic]

You probably need multiple endpoints.. or multiple consumers.

[amdonov]

Clebert,

I understand that I need multiple consumers. Is there a way to create multiple consumers with registering a URL multiple times? Would it be possible/advisable to change the PushConsumer to be backed by a pool?

Thanks,
Aaron

[jbertram]

One of the points of the HornetQ REST API is that there is no client implementation.  You simply need an HTTP library.  I don't ever see this changing so I don't see how we can support what your aiming for unless you can think of a way to make it 100% server-side.

[amdonov]

Justin,

My understanding is that the current PushConsumer creates a single consumer running on the HornetQ server that posts messages to my HTTP endpoint. It seems like it should be possible to pass pool size parameters when registering end points such that multiple message consumers are created on the HornetQ server which allows multiple concurrent requests to the target HTTP service. This is the behavior that I'm seeing now with multiple registrations of the same URL. The drawback is that I have to know in advance exactly how many consumers to create. In my current case, I'm registering the service endpoint 5 times which creates 5 message consumers. Ideally I'd just specifiy an upper limit like one does with a Message Driven Bean pool in JBoss. It would lead to much better utilization of resources. This would be 100% server-side and wouldn't require any changes to the target HTTP service.

Aaron

[jbertram]

I see what you're saying, and it doesn't appear that it would be too hard to implement.  That said, the server would be doing essentially exactly what you are doing.  There would be nothing "dynamic" about it.  If you told the push consumer you wanted 5 instances it would create 5 instances, period.  The number of instances wouldn't grow or shrink based on load or anything like that.  Since you mentioned MDBs it's worth noting that they work the same way.  When you supply a "maxSessions" activation configuration property value of e.g. 5 then the JCA layer creates 5 JMS sessions, period.  There is no dynamicity.  Also, I don't see any gains for resource utilization above what you are doing now.  The only change is whether you would do a for loop (or whatever) or pass a new parameter.

I'm not saying we can't (or won't) do it.  I'm just saying it might not be as big of a benefit as you appear to think.

Feel free to open a JIRA on this.

[amdonov]

Justin,

I think that it's worth implementing as part of registration even if the pool size remains constant. It's cleaner than submitting individual registration requests or multiple configuration files for durable subscribers. I'll create an issue in Jira.

Thanks,

Aaron