JBossMessaging performance
I've just completed a set of changes, mainly to enhance performance.
The good news is we're seeing some really quite good performance figures for non-persistent messages, especially considering we have a lot more avenues for optimization to explore yet :)
Just to give you a taste, where the JMS client and JMS server are in the same VM our performance already exceeds JBossMQ by a lot - this is for straight message sends, concurrent sends and receives etc.
On my laptop I'm seeing around 19000 msgs/sec for this use case.
I put this down largely to the smart copying I implemented some time back. With this only those parts of the message that need to be cloned are cloned lazily. This means that if a message is sent, then received in the same vm, and nothing is changed on it, then no cloning occurs - i.e. it's fast.
For inter-VM interactions, with JMS client in one VM, and JMS Server in other VM but on the same machine, performance is also very good.
Until a few days ago, on my laptop, I was typically seeing send rates for simple messages with no payloads of around 800 msgs/sec, this compares with JBossMQ which handles about 5000 on my box.
Now, we are seeing send rates aroundof 7500 msgs/sec, which already exceeds JBossMQ and, as I say, we have much more scope for further optimisation. Receive rates and concurrent send/receive rates are also very good.
This is not far off the capacity of 100MB/s ethernet.
In fact we have already exceeded JBossMQ by a long way in all tests involving non-persistent messages.
Persistent messages performance is not so good, but this is to be expected since we have a bunch of things to do to optimise the db access which we should be looking at very shortly.
To get the performance boost, it involved some changes to our read/writeExternal implementations, but the main thing was the creation of our own custom marshaller which gives us fine grained control over the wire format.
Previously for a simple send message, the message was being sent in a AOP Invocation object, sitting inside a Remoting InvocationRequest object which was all serialized and sent down the socket.
Then the response came back as a serialized InvocationResponse object in a lot of cases simply signifying a null response (this could be sent in one byte).
The overhead of the serialized objects added a lot of overhead when all we really wanted to send is the message (and a couple of other pieces of information).
This is what we now do. This accounts for the bulk of the performance gain.
As I mentioned earlier there are a bunch of other optimisations yet to be done, including:
1) Batching operations in same JDBC tx. - particularly useful for transactions.
2) Optimisations of the message and message ref tables.
3) Smart serialization of messages - this one deserves a bit more discussion.
For a persistent message being sent from a client in one VM to a server in another VM, then back to a receiving client in another VM, we have
a) Message is serialized just before it is sent.
b) Message is deserialized on receipt in server
c) Message is serialized just before it it persisted
d) Message is serialized just before being sent to the receiver.
e) Message is deserialized after receipt at the receiver.
When you consider that the message payload is opaque in the server, most of the above are unnecessary. In fact the message only needs to be serialized once (at send) and deserialized once (at receipt), it can be stored in the database as a byte[] (blob).
Implementing this should (hopefully) give a significant perf boost.
4) More fine tuning of readExternal, writeExternal
5) Various other bits and pieces.
Moreover, Clebert is also making some optimisations to JBossSerialization (which we are using for our serialization) so hopefully this will make a difference too.
