6 Replies Latest reply on Apr 11, 2014 7:24 AM by bresnicow

    min-large-message-size does not reflect number of message bytes

    bresnicow

      We are using HornetQ 2.2.3 with JMS messages.   When we send JMS messages to HornetQ queues, sometimes we get large message files created which we dont want.   Our min-large-message-size is set at 10 Meg.   But messages of only about 5 meg are getting large message files created.

       

      Far as i can tell, this is because Hornetq stores messages with a 2 byte encoding.  The large message files are double the size of the original text message.

       

      Is there a way to control how HornetQ stores large messages, so that it uses 1 ascii byte instead of 2 bytes?

       

      Shouldnt the min-large-message-size parameter reflect the number of characters in the message?

       

      Does this sound like a bug?

       

      Thanks,

      Bill R

        • 1. Re: min-large-message-size does not reflect number of message bytes
          ataylor

          thats quite an old version, do you see this happen with the latest release 2.4.1

          • 2. Re: min-large-message-size does not reflect number of message bytes
            jbertram

            HornetQ does use a 2-byte encoding so what you're seeing is expected (i.e. not a bug).  However, I can understand your confusion.  I think this is more of a user education issue.  I'll add something to the user manual about it.

             

            As far as I understand, we use a 2-byte encoding for (at least) the following reasons:

            1. Internationalization.
            2. Speed.
            • 3. Re: min-large-message-size does not reflect number of message bytes
              bresnicow

              We dont have 2.4 but I'll try it with 2.3.0 that we do have.

              • 4. Re: min-large-message-size does not reflect number of message bytes
                bresnicow

                Hi thanks for replying,

                 

                The problem is that the min-large-message-size parameter is off by a factor of two.  It should reflect the message size as number of characters regardless of encoding.  We issue JMS messages of about 5 Meg (ASCII encoded).   The min...size is set to 10 Meg.  But our messages get treated as large messages which is not what we want. Of course its a simple workaround to just double the min..size parameter.

                 

                Do you know where the code that handles the min-...size parameter is?

                Assuming this behavour is the same in the latest version I think an issue should be opened, shall I do so?

                 

                -thanks

                Bill R

                • 5. Re: min-large-message-size does not reflect number of message bytes
                  jbertram

                  The problem is that the min-large-message-size parameter is off by a factor of two.

                  I believe that only true in specific cases like yours, e.g. when 1-byte encoded strings are inserted into the message.  It isn't true for all cases.

                   

                  It should reflect the message size as number of characters regardless of encoding.

                  As I understand it, the encoding directly impacts the actual size of the message so I don't see how it can simply be disregarded here.  As I indicated previously, I see this as mainly a user-education issue, and I've updated the documentation to reflect that.

                   

                  I assume you're using a javax.jms.TextMessage in your application.  If you wish to avoid the 2-byte encoding for strings done by HornetQ then I believe you can use a javax.jms.BytesMessage instead to maintain the contents of the message byte for byte.

                   

                  You might also investigate using compress-large-messages.

                   

                  Do you know where the code that handles the min-...size parameter is?

                  The code that looks at the size of the message and determines whether or not it is large is in org.hornetq.core.client.impl.ClientProducerImpl.doSend.

                   

                  Assuming this behavour is the same in the latest version I think an issue should be opened, shall I do so?

                  You are certainly free to open a JIRA for this, but I think it's unlikely it will be changed.

                  • 6. Re: min-large-message-size does not reflect number of message bytes
                    bresnicow

                    Looks good,

                    thanks Justin

                     

                    Bill R