Using hornet core than JMS

[pnranjith]

I am planning to move from JMS to hornet core. In JMS we used ObjectMessage because our message has many properties. Since messaging with hornet core gives tremendous throughput(around 30k per sec), we are going to switch to core. But the problem faced is a replacement for JMS Object message( I know serialization kills time). But what alternative does hornetq core has to replace ObjectMessage in the JMS.

[timfox]

You can just do the serialisation/deserialisation yourself, something like:

ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);

ObjectOutputStream oos = new ObjectOutputStream(baos);

oos.writeObject(object);

oos.flush();

byte[] data = baos.toByteArray();

then send the byte[] in the core message.

And when receiving, just deserialize it yourself:

[zenzei2k]

Here is another example for converting to byte[], with some standard compression from JDK!

http://elegantengineering.blogspot.com/2008/12/data-compression-using-java-part-2.html

I tried with some objects of mine and the size gain of some of them was about 50%! For example.

Before Compression: (3374 bytes) com.test.TestVO@12ac982
After Compression: (1655 bytes) com.test.TestVO@1f20eeb

[zenzei2k]

Tim Fox escribió:

You can just do the serialisation/deserialisation yourself, something like:

ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);

ObjectOutputStream oos = new ObjectOutputStream(baos);

oos.writeObject(object);

oos.flush();

byte[] data = baos.toByteArray();

then send the byte[] in the core message.

And when receiving, just deserialize it yourself:

Hi Tim! Would't that be the same as using ObjectMessage, I mean the same poor performance? I ask because my knowledge of Serialization is not big, and looking at the source code of HornetQObjectMessage, what you suggest it's exactly what it's done there.  Or the HornetQ implementation of ObjectMessage is that an improvement compared to other messaging frameworks?

[timfox]

Nico Nico wrote:

Hi Tim! Would't that be the same as using ObjectMessage, I mean the same poor performance? I ask because my knowledge of Serialization is not big, and looking at the source code of HornetQObjectMessage, what you suggest it's exactly what it's done there.  Or the HornetQ implementation of ObjectMessage is that an improvement compared to other messaging frameworks?

This code is just using standard Java serialization.

The reason ObjectMessage is slow is because it uses Java serialization, not because the HornetQ implementation is slow.

All JMS implementations of ObjectMessage must use Java serialization - this is a requirement from the spec, it's nothing specific to HornetQ.

It's usually faster to marshall objects using your own wireformat. Serialization is generic and works with any object but the trade-off is it verbose and slow.

[zenzei2k]

Ok, sorry for the misunderstand. I thought you were telling Ranjith that the code you wrote in the second post was better than standard serialization (because of his question of what to use to replace ObjectMessage), when in fact is the standard serialization code.

[timfox]

AIUI Ranjith was asking how to serialize an object to and from a byte[] since he wants to use the core API, and the core API only deals with byte[].

[pnranjith]

Hi

I did as you said. But in creating the message back from the byte array I got, I am getting an exception:

java.io.StreamCorruptedException: invalid stream header: 0000036F
    at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:783)
    at java.io.ObjectInputStream.<init>(ObjectInputStream.java:280)
    at com.sit.q.JmsBase.getObjectFromClientMessage(JmsBase.java:255)
    at com.sit.q.ClientRequestListener.onMessage(ClientRequestListener.java:180)
    at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:767)
    at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:45)
    at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:885)
    at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)

This is how I did that:

    protected byte[] getByteMessage(Serializable objectMessage) throws IOException
    {
    ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);
    ObjectOutputStream oos;
    oos = new ObjectOutputStream(baos);
    oos.writeObject(objectMessage);
    oos.flush();

    byte[] data = baos.toByteArray();
    return data;
    }

    protected Object getObjectFromClientMessage(ClientMessage message) throws IOException, ClassNotFoundException
    {
    byte[] msg = new byte[message.getBodySize()];
    message.getBodyBuffer().getBytes(0, msg);
    ByteArrayInputStream bais = new ByteArrayInputStream(msg);
    ObjectInputStream ois;
    ois = new ObjectInputStream(bais);
    return ois.readObject();

    }

[timfox]

Look how it's done in HornetQObjectMessage, you can copy it from there:

http://anonsvn.jboss.org/repos/hornetq/trunk/src/main/org/hornetq/jms/client/HornetQObjectMessage.java

[pnranjith]

Hi Tim,

Thanks a lot.

I fixed that as per your post. Its running!.

Now the throughput I got is around 200 messages per second(with hornetq core). That is merely 100 messages per second more than that with JMS/hornetq implementation. Is there any other optimization we can do?. 

[timfox]

That question is almost impossible to answer since you haven't given enough information.

I've said this many, many times on this forum. Performance is a function of so many factors:

Size of messages

Are messages persistent/non persistent

Type of messages

What are you doing with the messages?

Server hardware

Storage hardware

Network Hardware

If you provide a *very small* self contained program I can tell you how to optimise it and explain why you get the performance you get. Otherwise the question is too general for me to provide any meaningful answer.

BTW, Core should not be twice as fast as JMS. Probably you're just not acknowledging your messages.