1 2 Previous Next 16 Replies Latest reply on May 17, 2006 11:39 AM by jaink

    Large 200mb messages

    jaink

      Hi,

      We are using JBoss Messaging with mysql as persistence store. We tried to send a 70mb message and it failed. Most of the FAQs and post related to large message support seems to be for JBossMQ. We have to eventually support 200mb messages.

      Has large message support changed in JBoss Messaging? Do we need to break the message up into smaller chunks? Can we use transaction to receive this group of messages as one transaction?

      Thanks.

        • 1. Re: Large 200mb messages
          timfox

          Can you tell me how it failed?

          Very large message support is currently scheduled for 1.2:

          See

          http://jira.jboss.com/jira/browse/JBMESSAGING-379

          http://jira.jboss.com/jira/browse/JBMESSAGING-153

          • 2. Re: Large 200mb messages
            jaink

            My apologizes for the wrong post. It was a heap size problem. 70mb works fine with 256m heap size. Only issue here is that our client machines have only 512m memory and run heavyweights like oracle.

            When will JBoss Messaging 1.2 be released?

            Thanks again for the reply.

            • 3. Re: Large 200mb messages
              timfox

              Yes, you can always compensate to some degree by increasing heap size, in your case this should work since your max message size is significantly less than available RAM, although for even larger messages this won't be the case.

              JBossMessaging release date is TBD, but "some time later this year" is all I can say right now.

              If this is a big priority for you we can consider putting it in an earlier release although I can't guarantee this will happen.

              • 4. Re: Large 200mb messages

                I would say it is quite important for my project too.

                Thanks
                Madhu

                • 5. Re: Large 200mb messages
                  jaink

                  Yes, it is. We are required to send huge images as part of a Medical project with JMS backbone ( for reliability) so early release (if possible) sounds great.

                  Thanks.

                  • 6. Re: Large 200mb messages
                    timfox

                    Ok, noted.

                    BTW You know you can vote for issues on JIRA if you want them bumped in priority?

                    • 7. Re: Large 200mb messages
                      timfox

                       

                      "jaink" wrote:
                      Yes, it is. We are required to send huge images as part of a Medical project with JMS backbone ( for reliability) so early release (if possible) sounds great.

                      Thanks.


                      Are the messages already compressed? There is also another task to enable compresssion at the transport layer, although obviously this is not going to help you if your images are already compressed (jpegs etc.)

                      • 8. Re: Large 200mb messages
                        jaink

                        Did it. Thanks.

                        • 9. Re: Large 200mb messages
                          jaink

                          Yes. We are compressing before sending.

                          Has anybody build their own message grouping (breaking up a huge message) and received the group as one entity using transactions? That seems to be our only solution at this point.

                          Thanks.

                          • 10. Re: Large 200mb messages
                            timfox

                            Transactions won't help since all messages in the transaction will be sent in one request - hence it will take up the same (actually slightly more) memory than sending a single message.

                            • 11. Re: Large 200mb messages
                              timfox

                              Instead if you break the message up into multiple messages and set JMSXGroupID and JMSXGroupSequence (see JMS 1.1 spec), and send the messages with delivery mode persistent to either a queue or a durable subscriber then you may have some luck.

                              Although you will still have the problem of the messages existing at the same time in memory on the server (how much memory do you have available on the server) - so you'll need to set the fullSize param on the queue/topic (see getting started guide) so you'll never have more than that many messages in memory at once.

                              • 12. Re: Large 200mb messages
                                jaink

                                Servers will be around a gig.

                                • 13. Re: Large 200mb messages
                                  timfox

                                  Since you won't be using transactions you will of course deal with the fact that your client or server could die after the set of messages was only partially sent.

                                  • 14. Re: Large 200mb messages
                                    jaink

                                    Thanks for the suggestions. Couple of questions

                                    1. JMSX properties seems to be optional. Can we assume that JMSXGroupid and JMSXGroupseq are supported by JBoss Messaging?

                                    2. How do transactions work with "fullSize, pageSize and downCacheSize" parameters? If transactions require all messages (as part of the transaction) to be in memory at the same time how does the fullsize restriction work?

                                    3. Not sure what is meant by "Since you won't be using transactions you will of course deal with the fact that your client or server could die after the set of messages was only partially sent."

                                    if we are using persisted messages with durable subscriber messages should be delivered on any client or server failure (once they recover). Is that correct?

                                    Apologize for the layman questions but we are still in discovery mode as far as JBoss Messaging is concerned.

                                    Thanks for your help again.


                                    1 2 Previous Next