6 Replies Latest reply on May 23, 2008 4:11 PM by jason hu

    Huge queue

    Kresimir Tonkovic Newbie

      I have a queue that sometimes fills up to a 1.000.000 messages. When restarting JBoss, it tries to load all of them into memory, which fails with OutOfMemoryException.

      I do not use selectors on this queue, and I see no other reason why would all these messages have to be in memory all the time. Is there a way to tell JBossMQ to load messages in chunks of, say 1000 messages, a time? Is there maybe any other solution? Is the new JBoss Messaging maybe more efficient in this scenario?

        • 1. workaround Re: Huge queue
          jason hu Newbie

          I am having the same issue. After I went through the source code, i found the problem. it tries to select all messages for a queue and then evaluate the memory mark, which definitely fail with millions of messages.

          the work around I have so far is to change the select sql in jdbc2.xml to only select first 100 message. for example for sql server is:
          select top 100 messageid, messageblob from jms_messages where destination=?

          i checked the jboss messaging, from the features it has, it seems handle this better, but have not got time to really try it. the version of my production jboss is 4.05,

          • 2. Re: Huge queue
            Kresimir Tonkovic Newbie

            I have tried limiting the number of messages to select as you suggested. But then MQ only loads that many messages, processes them and stops. A new select for for the next batch is never executed.

            Is there a way to force selection of a new batch? I have tried restarting the queue with twiddle:

            twiddle.sh invoke jboss.mq.destination:service=Queue,name=MyQueue stop
            twiddle.sh invoke jboss.mq.destination:service=Queue,name=MyQueue start

            The queue shows that it has a new batch of messages (QueueDepth), but it doesn't process them. In fact, I can post new messages into that queue, but the MDB is never executed again.

            • 3. Re: Huge queue
              Kresimir Tonkovic Newbie

              To answer my own question, I can make the Queue start processsing messages again by restarting delivery to the MDB:

              MDB=`twiddle.sh query "jboss.j2ee:service=EJB,plugin=invoker,binding=message-driven-bean,*"|grep MyQueue`
              twiddle.sh invoke $MDB stopDelivery
              twiddle.sh invoke $MDB startDelivery

              • 4. Re: Huge queue
                Adrian Brock Master

                See message chunking on the persistence manager config

                • 5. Re: Huge queue
                  jason hu Newbie

                  you need to set your message to expire after certain number processing, so the top 100 messages will be removed from the table after certain time. this is just a workaround, not a solution.

                  I think jboss has abandoned the this version jms impl, I saw the jboss messaging has covered this Huge Queue issue by parameters in destination file.
                  we are now waiting on jboss 5.0 release with jboss messaging.

                  • 6. Re: Huge queue
                    jason hu Newbie

                    set messagechuck =1 still won't work with really large queue. even the keys can blow the system itself