Simple JMS Queue - Out of Memory error

[jklem]

JBoss-3.0.3/JBossMQ/jdk1.4.1

Running a very light load using three message queues. After sending a few messages across (20-30) over a period of 10-15 minutes we start seeing this fatal error message in the output log...

>>>>>>>>>>>>
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start(Native Method)
at org.jboss.mq.il.oil.OILClientILService.start(OILClientILService.java:297)
at org.jboss.mq.Connection.startILService(Connection.java:837)
at org.jboss.mq.Connection.(Connection.java:244)
at org.jboss.mq.Connection.(Connection.java:264)
at org.jboss.mq.SpyConnection.(SpyConnection.java:59)
at org.jboss.mq.SpyConnectionFactory.createQueueConnection(SpyConnectionFactory.java:116)
...
>>>>>>>>>>>>

At this point, the JMS queues are rendered inoperable and the server must be killed an restarted. We are only seeing this problem on our production Sun platform to date. The app is also running on RedHat Linux and Win2K but we have not been able to repeat.

We have been unable to come up with anything to explain this behavior. Any ideas would be appreciated.

Thank you

-Jon Klem

[tommys]

Try it with Jboss3.0.4. There are some fixes in there for resource leak. You don't really have to upgrade your server, just copy jbossmq.jar from 3.0.4 to your app lib directory.

Also, check your own code for memory leaks. Are you keeping the messages somewhere but didn't delete them?

[adrian.brock]

Are you using selectors?
There is a fix in 3.0.4

Regards,
Adrian

[jklem]

We upgraded JBoss to 3.0.4 (we moved the whole server rather than just the MQ) and the results were disasterous. We got the Out of Memory error after sending about 5 or 6 total messages through our 3 Queues. We use 2 queues that recieve requests to process data and 1 queue that gathers status messages. For some reason it appears that the 3rd queue is the one that actually fails, or fails first. There is some evidence that the other 2 queues continue to function, at least for a time, even after the 3rd status queue is throwing out of memory errors.

We are not using selectors. The app sends a smallish xml message. Dispatching is handled in the appropriate MDB by using a few simple properties to determine which process will be performed.

-jk

[adrian.brock]

My first guess is you have some infinite loop?

Perhaps you can post a simple example that
reproduces the problem?

Regards,
Adrian

[yjing]

Is it the first time that you use JDK1.4.1 with jboss?

I ran into similar problem with jboss3.0.3/jdk1.4. It was caused by xml SAXParser who is using StringBuffer.setLength(0) in flushCharacters(). After changing StringBuffer.setLength(0) to new StringBuffer(), no "outofMemoryError" anymore.

Yan

[j.hedin]

I have also had problems with OutOfMemoryException and Jboss 3.0.4/jdk1.4.1_01.The JMS queue is used to propagate tiny XML messages. Could the errors not be related to JBossMQ, but to massive parsing of tiny XML messages? How did you fix the SAX parser? Do you use the Sun or Apache one. Did you recompile it?

P.S. To the JBoss group: Thanks for a wounderfull Appservere. D.S.

[j.hedin]

The key to my problems war raising the -Xmx option to the JVM. I have since then successfully sent 200k tiny XML messages though the JBossMQ to EJBs and last MySQL. Works perfectly (if it wasn't for the massive paging of my laptop). Tested on Linux. Havn't tried on the Solaris production enviroment yet.

[yjing]

My outofMemoryError with jboss3.0.3/jdk1.4.1_01 is caused by jdk1.4.1_01's StringBuffer.setLength(0). Just one 500K jms message will trigger this memory error. The receiver of the message uses JDOM to parse the 500k xml string.

It won't happen with Jdk1.3.5.

I changed JDOM's SAXHandle.java to not use stringBuffer.setLength(0). The problem is solved.


Yan