HornetQ Best Practices (for High Performance)

[vpothnis]

Hello,

I have a few questions related to best practices using HornetQ. The application that I am working on is using embedded Jetty. I intend to use the HornetQ as standalone server(s). The traffic/usage for this application is projected to be high with the peak usage of about 20,000 messages per second. There will be multiple instances of the application and a cluster of HornetQ servers. Given these,

1. What is the best approach for setting up Connections to HornetQ Servers from individual instances of the application - is there a need for a Connection pool?
2. If I go with simple MessageListeners to consume messages, is there any out-of-the-box support of pooling MessageListeners? The intention is to consume/process messages as fast as possible.

Any pointers to best practices related to the above will be very helpful.

Thanks!
Vinay

[timfox]

This really depends on your application architecture.

If you can describe it in some more detail we might be able to answer your question better.

[vpothnis]

Hello Tim,

Thanks for your response. I will try and detail out here.

Basically, I am trying to use HornetQ as the MoM. On one end, I have a cluster of embedded jetty web applications that will post messages to HornetQ. On the other end, I will have another cluster of embedded jetty web applications that will consume the messages and perform some activities based on the messages.

Now, since the rate of posting of messages is projected to be high (with a peak of around 20000 messages per second), i was wondering if HornetQ recommends any best practices to handle this kind of requirement.

1. What would be the recommended strategy for the applications posting message? Assuming that we have a cluster of HornetQ servers, should each of the message producing application open one connection or would there be any gains in opening multiple connections to the HornetQ servers?

2. What would be the recommended strategy for the consumers of the message, if we want to consume the messages as soon as possible? I do not want to use MDB pool. Should I go for a pool of MessageListeneres on each of the consumers? Would it be an overkill?
Spring provides a pooling solution for messgae listeners. But since I dont want to use Spring, I was wondering if HornetQ provides any pooling support for MessageListeners.

I hope this is adds a bit more clarity.

Thanks!
Vinay

[ataylor]

1. What would be the recommended strategy for the applications posting message? Assuming that we have a cluster of HornetQ servers, should each of the message producing application open one connection or would there be any gains in opening multiple connections to the HornetQ servers?

If you have a cluster of HornetQ Servers then i would recommend creating at least enough connections to match the number of nodes on the server. If you have only one node then the messages will have to be re distributed from the one node to the other. The speed messages can be sent via a single connection depends on a few things bandwidth, flow control, message consumption etc. I would suggest running some tests with your application to see how it copes. Bear in mind that there are other things that alter performance such as are you using block or non block on send, are you using a transaction, the size of the message itself, etc etc. All these things are covered in the User Manual

2. What would be the recommended strategy for the consumers of the message, if we want to consume the messages as soon as possible? I do not want to use MDB pool. Should I go for a pool of MessageListeneres on each of the consumers? Would it be an overkill?
Spring provides a pooling solution for messgae listeners. But since I dont want to use Spring, I was wondering if HornetQ provides any pooling support for MessageListeners.

Again this depends on how fast your consumers consume the messages, HornetQ doesnt support pooling of listeners but this should be pretty easy for you to implement. Again, things like the acklnowledgement mode will also effect performance, again take a look in the User manual

[timfox]

Web applications are a little tricky since the stateless web paradigm doesn't fit very well for writing messaging clients.

You clearly don't want to be creating a new connection every time a web request comes in, so you'll have to think about pooling connections.

I don't know enough about Jetty to know if it contains this functionality or it has hooks allowing you to start up or shutdown pools on server startup/shutdown, but in any case these points aren't specific to HornetQ but would apply to using any messaging provider from a web application, so I guess you just need to look up what is the best practice in this area. This is a very common pattern.

[vpothnis]

Andy and Tim,

Thanks very much for your inputs!

-Vinay