Performance Testing HornetQ

[brgorrie]

Hi,

Has anyone done performance testing (or created a set of performance tests) of HornetQ?

The reason I'm asking is that I am currently putting a set of performance tests together and don't want to reinvent the wheel if I can help it.

As an FYI, here is my first pass at basic sending/recieving metrics (the percentages indicate X% of calls were less then or equal to the time).

{code}

[main] 09:53:37,188 INFO [org.hornetq.jms.tests.QueueTest]  [org.hornetq.jms.tests.QueueTest:Metrics:BaseQueueTestRecieve] [ 7121 ms ] 5000 @ 702.149 /sec, Avg: 0.611816 ms, Min: 0.440846 ms, Max: 10.611608 ms, Median: 0.536653 ms, Mode: 0.508169 ms @ 3 , 50%: 0.536652 ms, 75%: 0.577537 ms, 90%: 0.809641 ms, 95%: 0.987134 ms, 99%: 1.940366 ms
[main] 09:53:37,195 INFO [org.hornetq.jms.tests.QueueTest]  [org.hornetq.jms.tests.QueueTest:Metrics:BaseQueueTestSend] [ 7128 ms ] 5000 @ 701.459 /sec, Avg: 0.788865 ms, Min: 0.54222 ms, Max: 120.14141 ms, Median: 0.640311 ms, Mode: 0.620933 ms @ 3 , 50%: 0.640302 ms, 75%: 0.688609 ms, 90%: 0.981365 ms, 95%: 1.171842 ms, 99%: 2.164544 ms
[main] 09:53:43,850 INFO [org.hornetq.jms.tests.QueueTest]  [org.hornetq.jms.tests.QueueTest:Metrics:BaseQueueTestRecieve] [ 6663 ms ] 5000 @ 750.413 /sec, Avg: 0.588192 ms, Min: 0.439769 ms, Max: 6.518101 ms, Median: 0.535902 ms, Mode: 0.517335 ms @ 3 , 50%: 0.535901 ms, 75%: 0.572536 ms, 90%: 0.66436 ms, 95%: 0.905616 ms, 99%: 1.425827 ms
[main] 09:53:43,855 INFO [org.hornetq.jms.tests.QueueTest]  [org.hornetq.jms.tests.QueueTest:Metrics:BaseQueueTestSend] [ 6669 ms ] 5000 @ 749.738 /sec, Avg: 0.720151 ms, Min: 0.541465 ms, Max: 10.521575 ms, Median: 0.640314 ms, Mode: 0.576738 ms @ 2 , 50%: 0.640309 ms, 75%: 0.680393 ms, 90%: 0.81478 ms, 95%: 1.150456 ms, 99%: 2.194207 ms

{code}

And here are the changes I made to the test case in the QueueTest class to produce the output.

{code}

    /**
      * The simplest possible queue test.
      */
     public void testQueue() throws Exception {
         Connection conn = null;

         try {
             conn = JMSTestCase.cf.createConnection();

             Session s = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
             MessageProducer p = s.createProducer(HornetQServerTestCase.queue1);
             MessageConsumer c = s.createConsumer(HornetQServerTestCase.queue1);
             conn.start();

             TextMessage textMessage = s.createTextMessage("payload");

             String metricsIdSend = metrics.registerForMetricsLogging("BaseQueueTestSend", getClass());
             String metricsIdRecieve = metrics.registerForMetricsLogging("BaseQueueTestRecieve", getClass());

             for (int j = 0; j < 20; j++) {
                 for (int i = 0; i < 5000; i++) {
                     long startTime = System.nanoTime();
                     try {
                         p.send(textMessage);
                     } finally {
                         metrics.addTimeToMetrics(metricsIdSend, System.nanoTime(), startTime);
                     }
                     TextMessage m = null;
                     startTime = System.nanoTime();
                     try {
                         m = (TextMessage) c.receive();
                     } finally {
                         metrics.addTimeToMetrics(metricsIdRecieve, System.nanoTime(), startTime);
                     }
                     if (m != null) {
                         ProxyAssertSupport.assertEquals("payload", m.getText());
                     }
                 }
                 metrics.outputStatistics();
             }

             p.send(textMessage);
             TextMessage m = (TextMessage) c.receive();
             ProxyAssertSupport.assertEquals("payload", m.getText());

         } finally {
             if (conn != null) {
                 conn.close();
             }
         }
     }

{code}

Thoughts?

Cheers,

Brian.

[brgorrie]

And my apologies, it looks like my attempt to use the wiki markup didn't work as intended.

Cheers,

Brian.

[timfox]

Brian,

We have a performance testing framework that we wrote some time back for JBoss Messaging:

http://anonsvn.jboss.org/repos/messaging/projects/perf/

Also we're going to be using SpecJMS some time in the not too distant future:

http://www.spec.org/jms2007/

There's also a perf testing framework that the activemq guys created:

http://jmstester.fusesource.org/documentation/index.html

I'd recommend not re-inventing the wheel.

Simplistic tests like the one you posted - just sending messages in a blocking fashion one after another really don't demonstrate much of interest - you're basically just measuring your network latency.

[brgorrie]

Hi Tim,

Thank you very much for your response. I'll pull apart the test suites and see what I can find out about Hornet Q's performance.  Will let you know if anything interesting turns up from my testing.

Yep that was a very simplistic test, interstingly if I did the 5000 sends followed by the 5000 recieves rather then doing a send followed immediately by a recieve 5000 times I doubled the throughput of that test.  Making it concurrent as well also moved the upper limit higher again.  Then increasing the number of threads doing sending and recieving also moved the throughput higher still.  About then I realised I was reinventing the wheel ;-).

And yes I'm seriously allergic to reinventing the wheel ;-) its why I presented the simple example instead of a complex one ;-).

Cheers,

Brian.