Huge queue

[kresho]

I have a queue that sometimes fills up to a 1.000.000 messages. When restarting JBoss, it tries to load all of them into memory, which fails with OutOfMemoryException.

I do not use selectors on this queue, and I see no other reason why would all these messages have to be in memory all the time. Is there a way to tell JBossMQ to load messages in chunks of, say 1000 messages, a time? Is there maybe any other solution? Is the new JBoss Messaging maybe more efficient in this scenario?

[jasonhu919]

I am having the same issue. After I went through the source code, i found the problem. it tries to select all messages for a queue and then evaluate the memory mark, which definitely fail with millions of messages.

the work around I have so far is to change the select sql in jdbc2.xml to only select first 100 message. for example for sql server is:
select top 100 messageid, messageblob from jms_messages where destination=?

i checked the jboss messaging, from the features it has, it seems handle this better, but have not got time to really try it. the version of my production jboss is 4.05,

[kresho]

I have tried limiting the number of messages to select as you suggested. But then MQ only loads that many messages, processes them and stops. A new select for for the next batch is never executed.

Is there a way to force selection of a new batch? I have tried restarting the queue with twiddle:

twiddle.sh invoke jboss.mq.destination:service=Queue,name=MyQueue stop
twiddle.sh invoke jboss.mq.destination:service=Queue,name=MyQueue start

The queue shows that it has a new batch of messages (QueueDepth), but it doesn't process them. In fact, I can post new messages into that queue, but the MDB is never executed again.

[kresho]

To answer my own question, I can make the Queue start processsing messages again by restarting delivery to the MDB:

MDB=`twiddle.sh query "jboss.j2ee:service=EJB,plugin=invoker,binding=message-driven-bean,*"|grep MyQueue`
twiddle.sh invoke $MDB stopDelivery
twiddle.sh invoke $MDB startDelivery

[adrian.brock]

See message chunking on the persistence manager config

[jasonhu919]

you need to set your message to expire after certain number processing, so the top 100 messages will be removed from the table after certain time. this is just a workaround, not a solution.

I think jboss has abandoned the this version jms impl, I saw the jboss messaging has covered this Huge Queue issue by parameters in destination file.
we are now waiting on jboss 5.0 release with jboss messaging.

[jasonhu919]

set messagechuck =1 still won't work with really large queue. even the keys can blow the system itself