[timfox]

So on your 8Xeon, 32GB RAM, 15,000 rpm hard-disk system you mentioned you measured 237 syncs / sec

That sounds right, since 15000/60 = 250 revolutions per sec, and I guess it has write cache disabled.

[mreasy]

Nice article, thx. Interesting yet worrying results. Maybe an approach of forcing out the data using timeslices would increase the overall throughput, while increasing the response-times for some single messages - sth. like: wait 5ms for messages, then force data out for all of them. First message would have to wait longer, but throughput raises when looking at multiple messages.


The following is your SyncSpeedTest modified slightly to work on a MappedByteBuffer. By using MappedByteBuffer#force I also make the data being written to disk with the same guarantees as FileChannel#force(): http://pastebin.com/m5159d7e3

The throughput differs significantly on the windows system, which can also be observed by looking at the disk-activity (0.7MB/s vs. 5MB/s), but unfortunately it has no effect on Linux and Solaris; so no real benefit.
OS-write-cache was on in both tests on the Windows system.

D:\test\speedtest>java -cp . -Xmx1G SyncSpeedTest
Measuring
Rate of 42.96167013492649 syncs per sec
D:\test\speedtest>java -cp . -Xmx1G SyncSpeedTestMapped
Measuring
Rate of 309.4871097409859 syncs per sec

[timfox]

At the end of the day, this is a hardware limitation, not something specific to HornetQ (take a look how MySQL or ActiveMQ performs).

Waiting x ms before syncing is something we can consider for NIO (we actually already do something similar for ASYNCIO).

For GA we'll make sure we have the best possible performance, but fundamentally we can never go faster than what the hardware allows, so in a way it shouldn't be worrying, there's nothing we can do to increase a disk's maximum hardware sync rate, so this is really to be expected.

The way expensive disks get around this is to use a battery backed / NVRAM cache to buffer the syncs but still maintain data durability in event of failure.

[steffi]

I'm curious as to what if any different a solid state disk drive makes with the test? I do have an Intel X-M25 on my MacBook Pro that I'd be willing to test for you if you explain how I should complete the benchmark.

[steffi]

So when I change warmup to 10k and num to 100k here's what I get on my MacBook Pro

[java] INFO: average: 814.35 msg/s (100000 messages in 122.80s)

[steffi]

When I do the same in JMS here's what I get

[java] INFO: average: 938.37 msg/s (100000 messages in 106.57s)

"steffi2" wrote:
So when I change warmup to 10k and num to 100k here's what I get on my MacBook Pro

[java] INFO: average: 814.35 msg/s (100000 messages in 122.80s)

[steffi]

And here's SyncSpeed test on my machine

audrey:jars robert$ java -cp ./hornetq-core.jar org.hornetq.core.journal.impl.SyncSpeedTest
Oct 18, 2009 8:01:50 AM org.hornetq.core.logging.impl.JULLogDelegate info
INFO: ******* Starting file sync speed test *******
Oct 18, 2009 8:01:51 AM org.hornetq.core.logging.impl.JULLogDelegate info
INFO: Measuring
Oct 18, 2009 8:01:52 AM org.hornetq.core.logging.impl.JULLogDelegate info
INFO: Rate of 7372.2102231821455 syncs per sec
Oct 18, 2009 8:01:52 AM org.hornetq.core.logging.impl.JULLogDelegate info
INFO: ****** test complete *****