8 Replies Latest reply on Jan 20, 2010 3:34 PM by vpothnis

HornetQ Performance Benchmarks Available?

vpothnis Jan 20, 2010 1:32 PM

Hello,

Are the performance benchmark numbers available for HornetQ yet?

In the meanwhile, I did some simple tests on our own. I read this post only later: http://community.jboss.org/message/519156#519156

I would like to describe the tests and the numbers I got. Please advice if you see anything amiss.

The Setup:

Box 1: HornetQ Server 1

Box 2: HornetQ Server 2

Box 3: Simple Java Program (Producer)

Box 4: Simple Java Program (Consumer)

We have a cluster of the 2 HornetQ Servers via discovery groups. And the HornetQ servers are set up for ZERO PERSISTENCE.

The Consumer program sets up 6 message listeners that are load balanced across the 2 nodes of the cluster.

The Producer program can spawn multiple threads that can post messages to a queue. (PRE_ACKNOWLEDGE mode is used). Each thread sends 10,000 messages in a loop.

Case 1: In this case, the multiple threads of the Producer share a single MessageProducer instance. (Synchronized access). With this setup, I could get around 5000 messages per second. Increasing the number of threads had no effect. (Probably because of the shared MessageProducer)

Case 2: In the case, each thread had its own instance of MessageProducer. With this set up, I could get around 10,000 messages per second. Increasing the number of threads only had a marginal increase. It was as though, it plateaued at 10,000 messages per second.

Case 3: In this case, I created multiple instances of the Producer program. Each instance of the program could spawn multiple threads and each thread had its own instance of the MessageProducer.

Now, with 2 instance of Producer program each running about 30 threads, I could get about 20000 messages per second.

So, we were kind of confused with the numbers. It would be greatly helpful if HornetQ published its performance numbers so that there is some sort of reference. Also, it would be helpful if there is any explanation for Case 2. With a single instance of the program, the messages per second did not increase with the increase in number of threads.

Thanks

Vinay

1. Re: HornetQ Performance Benchmarks Available?

timfox Jan 20, 2010 2:17 PM (in response to vpothnis)

There will be some publicly available performance figures in the next few weeks.

There's really no point in commenting on your figures.

Performance depends on many factors including the hardware you are using, the OS you are using, how you have configured and tuned the system, and your test program.

None of the above information you provided in your post.
Actions
2. Re: HornetQ Performance Benchmarks Available?

timfox Jan 20, 2010 2:19 PM (in response to timfox)

Quoting absolute performance figure is really rather meaningless, since it's highly unlikely you are using the exact same hardware or OS configuration.
Actions
3. Re: HornetQ Performance Benchmarks Available?

vpothnis Jan 20, 2010 2:34 PM (in response to timfox)

Well, As I clearly stated, I wasn't expecting any affirmation/confirmation on the numbers. I understand that a number of factors impact the performance.
That's the reason, I wanted to check if there were any published numbers and the criteria.

Hope that it is available soon.
Actions
4. Re: HornetQ Performance Benchmarks Available?

timfox Jan 20, 2010 2:39 PM (in response to vpothnis)

You asked me to comment on the observations of your number of producers and threads.

I cannot possibly do that unless you provide your test program, an explanation of your network topology, and the specifications of the servers you are running with at the least.

And please, don't get impatient or rude.

We work long hours and weekends so you can have HornetQ for free.

When you have a sour attitude it just makes us not want to help you.
Actions
5. Re: HornetQ Performance Benchmarks Available?

vpothnis Jan 20, 2010 2:47 PM (in response to timfox)

Oh! Absolutely not. I am sorry If my comment seemed rude. That was not the intention at all.
I totally understand the effort and commitment that goes in. I just wanted to clarify that I did not expect any clearcut answers.

I hope I was able to convey this properly.
Actions
6. Re: HornetQ Performance Benchmarks Available?

vpothnis Jan 20, 2010 2:48 PM (in response to vpothnis)

And yes, I am not impatient - just eager.
Actions
7. Re: HornetQ Performance Benchmarks Available?

timfox Jan 20, 2010 3:25 PM (in response to vpothnis)

Vinay,

Apologies for being touchy - I have been working since 5am;)

Regarding performance benchmarks:

My philosophy with benchmarks is that I don't want to publish any benchmarks that aren't either peer reviewed or completely reproducible.

Quite often you'll see other messaging systems producing figures like "We can do X thousand messages per second" - I consider these kinds of results to be really meaningless, really just marketing fluff.

Any performance figures should be published with completely reproducible, publicly available test programs, and should state the complete hardware/software/OS configuration, so they can be debated properly in the public domain

So until we can produce them to this high standard I'm not going to make any quantitative claims. But rest assured, this is all in the pipe line.

Having said, I can say that the perf results we have got so far are pretty astonishingly. So watch this space
Actions
8. Re: HornetQ Performance Benchmarks Available?

vpothnis Jan 20, 2010 3:34 PM (in response to timfox)

Tim,

Please accept my apologies as well - I will be more careful in choosing my words
Most importantly, many thanks for bringing this to a closure

I agree with your thoughts on the benchmark numbers. Glad to hear about your initial results! And I will definitely be watching for more on this.

-Thanks
Vinay
Actions

Go to original post