-
1. Re: Using hornet core than JMS
timfox Apr 23, 2010 7:46 AM (in response to pnranjith)You can just do the serialisation/deserialisation yourself, something like:
ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(object);
oos.flush();
byte[] data = baos.toByteArray();
then send the byte[] in the core message.
And when receiving, just deserialize it yourself:
-
2. Re: Using hornet core than JMS
zenzei2k Apr 23, 2010 1:14 PM (in response to timfox)Here is another example for converting to byte[], with some standard compression from JDK!
http://elegantengineering.blogspot.com/2008/12/data-compression-using-java-part-2.html
I tried with some objects of mine and the size gain of some of them was about 50%! For example.
Before Compression: (3374 bytes) com.test.TestVO@12ac982
After Compression: (1655 bytes) com.test.TestVO@1f20eeb -
3. Re: Using hornet core than JMS
zenzei2k Apr 24, 2010 11:33 AM (in response to timfox)Tim Fox escribió:
You can just do the serialisation/deserialisation yourself, something like:
ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);
ObjectOutputStream oos = new ObjectOutputStream(baos);
oos.writeObject(object);
oos.flush();
byte[] data = baos.toByteArray();
then send the byte[] in the core message.
And when receiving, just deserialize it yourself:
Hi Tim! Would't that be the same as using ObjectMessage, I mean the same poor performance? I ask because my knowledge of Serialization is not big, and looking at the source code of HornetQObjectMessage, what you suggest it's exactly what it's done there. Or the HornetQ implementation of ObjectMessage is that an improvement compared to other messaging frameworks?
-
4. Re: Using hornet core than JMS
timfox Apr 25, 2010 5:33 AM (in response to zenzei2k)Nico Nico wrote:
Hi Tim! Would't that be the same as using ObjectMessage, I mean the same poor performance? I ask because my knowledge of Serialization is not big, and looking at the source code of HornetQObjectMessage, what you suggest it's exactly what it's done there. Or the HornetQ implementation of ObjectMessage is that an improvement compared to other messaging frameworks?
This code is just using standard Java serialization.
The reason ObjectMessage is slow is because it uses Java serialization, not because the HornetQ implementation is slow.
All JMS implementations of ObjectMessage must use Java serialization - this is a requirement from the spec, it's nothing specific to HornetQ.
It's usually faster to marshall objects using your own wireformat. Serialization is generic and works with any object but the trade-off is it verbose and slow.
-
5. Re: Using hornet core than JMS
zenzei2k Apr 25, 2010 10:13 AM (in response to timfox)Ok, sorry for the misunderstand. I thought you were telling Ranjith that the code you wrote in the second post was better than standard serialization (because of his question of what to use to replace ObjectMessage), when in fact is the standard serialization code.
-
6. Re: Using hornet core than JMS
timfox Apr 26, 2010 6:56 AM (in response to zenzei2k)AIUI Ranjith was asking how to serialize an object to and from a byte[] since he wants to use the core API, and the core API only deals with byte[].
-
7. Re: Using hornet core than JMS
pnranjith Apr 27, 2010 4:53 AM (in response to timfox)Hi
I did as you said. But in creating the message back from the byte array I got, I am getting an exception:
java.io.StreamCorruptedException: invalid stream header: 0000036F
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:783)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:280)
at com.sit.q.JmsBase.getObjectFromClientMessage(JmsBase.java:255)
at com.sit.q.ClientRequestListener.onMessage(ClientRequestListener.java:180)
at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:767)
at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:45)
at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:885)
at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)This is how I did that:
protected byte[] getByteMessage(Serializable objectMessage) throws IOException
{
ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);
ObjectOutputStream oos;
oos = new ObjectOutputStream(baos);
oos.writeObject(objectMessage);
oos.flush();byte[] data = baos.toByteArray();
return data;
}protected Object getObjectFromClientMessage(ClientMessage message) throws IOException, ClassNotFoundException
{
byte[] msg = new byte[message.getBodySize()];
message.getBodyBuffer().getBytes(0, msg);
ByteArrayInputStream bais = new ByteArrayInputStream(msg);
ObjectInputStream ois;
ois = new ObjectInputStream(bais);
return ois.readObject();}
-
8. Re: Using hornet core than JMS
timfox Apr 27, 2010 5:02 AM (in response to pnranjith)Look how it's done in HornetQObjectMessage, you can copy it from there:
-
9. Re: Using hornet core than JMS
pnranjith Apr 27, 2010 6:41 AM (in response to timfox)Hi Tim,
Thanks a lot.
I fixed that as per your post. Its running!.
Now the throughput I got is around 200 messages per second(with hornetq core). That is merely 100 messages per second more than that with JMS/hornetq implementation. Is there any other optimization we can do?.
-
10. Re: Using hornet core than JMS
timfox Apr 27, 2010 7:00 AM (in response to pnranjith)That question is almost impossible to answer since you haven't given enough information.
I've said this many, many times on this forum. Performance is a function of so many factors:
Size of messages
Are messages persistent/non persistent
Type of messages
What are you doing with the messages?
Server hardware
Storage hardware
Network Hardware
If you provide a *very small* self contained program I can tell you how to optimise it and explain why you get the performance you get. Otherwise the question is too general for me to provide any meaningful answer.
BTW, Core should not be twice as fast as JMS. Probably you're just not acknowledging your messages.