10 Replies Latest reply on Apr 27, 2010 7:00 AM by Tim Fox

    Using hornet core than JMS

    Ranjith PN Novice

      I am planning to move from JMS to hornet core. In JMS we used ObjectMessage because our message has many properties. Since messaging with hornet core gives tremendous throughput(around 30k per sec), we are going to switch to core. But the problem faced is a replacement for JMS Object message( I know serialization kills time). But what alternative does hornetq core has to replace ObjectMessage in the JMS.

        • 1. Re: Using hornet core than JMS
          Tim Fox Master

          You can just do the serialisation/deserialisation yourself, something like:

           

          ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);

           

          ObjectOutputStream oos = new ObjectOutputStream(baos);

           

          oos.writeObject(object);

           

          oos.flush();

           

          byte[] data = baos.toByteArray();

           

           

          then send the byte[] in the core message.

           

          And when receiving, just deserialize it yourself:

          • 2. Re: Using hornet core than JMS
            Nico Nico Newbie

            Here is another example for converting to byte[], with some standard compression from JDK!

             

            http://elegantengineering.blogspot.com/2008/12/data-compression-using-java-part-2.html

             

            I tried with some objects of mine and the size gain of some of them was about 50%! For example.

            Before Compression: (3374 bytes) com.test.TestVO@12ac982
            After Compression: (1655 bytes) com.test.TestVO@1f20eeb

            • 3. Re: Using hornet core than JMS
              Nico Nico Newbie

              Tim Fox escribió:

               

              You can just do the serialisation/deserialisation yourself, something like:

               

              ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);

               

              ObjectOutputStream oos = new ObjectOutputStream(baos);

               

              oos.writeObject(object);

               

              oos.flush();

               

              byte[] data = baos.toByteArray();

               

               

              then send the byte[] in the core message.

               

              And when receiving, just deserialize it yourself:

              Hi Tim! Would't that be the same as using ObjectMessage, I mean the same poor performance? I ask because my knowledge of Serialization is not big, and looking at the source code of HornetQObjectMessage, what you suggest it's exactly what it's done there.  Or the HornetQ implementation of ObjectMessage is that an improvement compared to other messaging frameworks?

              • 4. Re: Using hornet core than JMS
                Tim Fox Master

                Nico Nico wrote:

                 

                Hi Tim! Would't that be the same as using ObjectMessage, I mean the same poor performance? I ask because my knowledge of Serialization is not big, and looking at the source code of HornetQObjectMessage, what you suggest it's exactly what it's done there.  Or the HornetQ implementation of ObjectMessage is that an improvement compared to other messaging frameworks?

                This code is just using standard Java serialization.

                 

                The reason ObjectMessage is slow is because it uses Java serialization, not because the HornetQ implementation is slow.

                 

                All JMS implementations of ObjectMessage must use Java serialization - this is a requirement from the spec, it's nothing specific to HornetQ.

                 

                It's usually faster to marshall objects using your own wireformat. Serialization is generic and works with any object but the trade-off is it verbose and slow.

                • 5. Re: Using hornet core than JMS
                  Nico Nico Newbie

                  Ok, sorry for the misunderstand. I thought you were telling Ranjith that the code you wrote in the second post was better than standard serialization (because of his question of what to use to replace ObjectMessage), when in fact is the standard serialization code.

                  • 6. Re: Using hornet core than JMS
                    Tim Fox Master

                    AIUI Ranjith was asking how to serialize an object to and from a byte[] since he wants to use the core API, and the core API only deals with byte[].

                    • 7. Re: Using hornet core than JMS
                      Ranjith PN Novice

                      Hi

                      I did as you said. But in creating the message back from the byte array I got, I am getting an exception:

                       

                      java.io.StreamCorruptedException: invalid stream header: 0000036F
                          at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:783)
                          at java.io.ObjectInputStream.<init>(ObjectInputStream.java:280)
                          at com.sit.q.JmsBase.getObjectFromClientMessage(JmsBase.java:255)
                          at com.sit.q.ClientRequestListener.onMessage(ClientRequestListener.java:180)
                          at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:767)
                          at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:45)
                          at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:885)
                          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:96)
                          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
                          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
                          at java.lang.Thread.run(Thread.java:619)

                       

                      This is how I did that:

                       

                          protected byte[] getByteMessage(Serializable objectMessage) throws IOException
                          {
                          ByteArrayOutputStream baos = new ByteArrayOutputStream(1024);
                          ObjectOutputStream oos;
                          oos = new ObjectOutputStream(baos);
                          oos.writeObject(objectMessage);
                          oos.flush();

                       

                          byte[] data = baos.toByteArray();
                          return data;
                          }

                       

                          protected Object getObjectFromClientMessage(ClientMessage message) throws IOException, ClassNotFoundException
                          {
                          byte[] msg = new byte[message.getBodySize()];
                          message.getBodyBuffer().getBytes(0, msg);
                          ByteArrayInputStream bais = new ByteArrayInputStream(msg);
                          ObjectInputStream ois;
                          ois = new ObjectInputStream(bais);
                          return ois.readObject();

                          }

                      • 9. Re: Using hornet core than JMS
                        Ranjith PN Novice

                        Hi Tim,

                        Thanks a lot.

                        I fixed that as per your post. Its running!.

                        Now the throughput I got is around 200 messages per second(with hornetq core). That is merely 100 messages per second more than that with JMS/hornetq implementation. Is there any other optimization we can do?. 

                        • 10. Re: Using hornet core than JMS
                          Tim Fox Master

                          That question is almost impossible to answer since you haven't given enough information.

                           

                          I've said this many, many times on this forum. Performance is a function of so many factors:

                           

                          Size of messages

                          Are messages persistent/non persistent

                          Type of messages

                          What are you doing with the messages?

                          Server hardware

                          Storage hardware

                          Network Hardware

                           

                          If you provide a *very small* self contained program I can tell you how to optimise it and explain why you get the performance you get. Otherwise the question is too general for me to provide any meaningful answer.

                           

                          BTW, Core should not be twice as fast as JMS. Probably you're just not acknowledging your messages.