java.lang.OutOfMemoryError in 3.0.4

[cholliday]

I got this error in the server log

2002-11-26 13:34:52,399 ERROR [STDERR] java.lang.OutOfMemoryError

The server was running fine since I first installed JBoss 3.0.4 (Two weeks) until now. This seems to be the same problem I had with all the previous versions of JBoss. Looking back through my production log it seems that it had the problem first sending topic messages over a nundurable topic that uses selectors but then the queues also started having the problem. I didn't have time to try much out since it is a production box. I tried to shut it down but every thing didn't shutdown so I rebooted. JBoss wouldn't come up properly then because it was dying trying to recover a message, so I ended up deleting the file that represented that message and rebooted again. Then it worked.

I am looking for some/any help I can get with this. It is urgent because I am running in production. I was hoping this was fixed in 3.0.4

[garyg]

My first idea would be to go back and find a version of JBoss that didn't have this problem?

If you never had a working stable version, print the entire stack trace and what ever other info you have ... that's a bit sparse below.

[cholliday]

Never had a truly stable environment.

This is the stack trace from the applet.

java.lang.NoClassDefFoundError

at org.jboss.mq.SpyJMSException.(SpyJMSException.java:67)

at org.jboss.mq.Connection.sendToServer(Connection.java:1127)

at org.jboss.mq.SpySession.sendMessage(SpySession.java:654)

at org.jboss.mq.SpyQueueSender.internalSend(SpyQueueSender.java:118)

at org.jboss.mq.SpyQueueSender.send(SpyQueueSender.java:84)

at com.uss.irv.cr84.sys.jms.QueueWriter.publish(QueueWriter.java:119)

at com.uss.irv.cr84.sys.jms.QueueWriter.send(QueueWriter.java:80)

at com.uss.irv.cr84.rollshop.grind.GrinderQMsgWriter.broadcast(GrinderQMsgWriter.java:39)

at com.uss.irv.cr84.rollshop.grind.GrinderPanel.sendPreGrindTests(GrinderPanel.java:2159)

at com.uss.irv.cr84.rollshop.grind.GrinderPanel.handleClickEvents(GrinderPanel.java:2426)

at com.uss.irv.cr84.rollshop.grind.GrinderPanel.handleEvents(GrinderPanel.java:2404)

at com.uss.irv.cr84.rollshop.RollshopPanel.actionPerformed(RollshopPanel.java:102)

at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)

at javax.swing.AbstractButton$ForwardActionEvents.actionPerformed(Unknown Source)

at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)

at javax.swing.DefaultButtonModel.setPressed(Unknown Source)

at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(Unknown Source)

at java.awt.Component.processMouseEvent(Unknown Source)

at java.awt.Component.processEvent(Unknown Source)

at java.awt.Container.processEvent(Unknown Source)

at java.awt.Component.dispatchEventImpl(Unknown Source)

at java.awt.Container.dispatchEventImpl(Unknown Source)

at java.awt.Component.dispatchEvent(Unknown Source)

at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)

at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)

at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)

at java.awt.Container.dispatchEventImpl(Unknown Source)

at java.awt.Component.dispatchEvent(Unknown Source)

at java.awt.EventQueue.dispatchEvent(Unknown Source)

at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown Source)

at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)

at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

at java.awt.EventDispatchThread.pumpEvents(Unknown Source)

at java.awt.EventDispatchThread.run(Unknown Source)



I don't have a stack trace for the server I just get this in the server log.

2002-12-02 12:32:04,625 ERROR [STDERR] java.lang.OutOfMemoryError

[cholliday]

I put some better stack traces in and I get this.

On the client (Applet)

java.lang.ExceptionInInitializerError

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Unknown Source)

at org.jboss.util.NestedThrowable$Util.class$(NestedThrowable.java:27)

at org.jboss.util.NestedThrowable$Util.(NestedThrowable.java:101)

at org.jboss.mq.SpyJMSException.(SpyJMSException.java:67)

at org.jboss.mq.Connection.pingServer(Connection.java:1026)

at org.jboss.mq.Connection$PingTask.run(Connection.java:1183)

at EDU.oswego.cs.dl.util.concurrent.ClockDaemon$RunLoop.run(ClockDaemon.java:364)

at java.lang.Thread.run(Unknown Source)

Caused by: java.security.AccessControlException: access denied (java.util.PropertyPermission org.jboss.util.NestedThrowable.parentTraceEnabled read)

at java.security.AccessControlContext.checkPermission(Unknown Source)

at java.security.AccessController.checkPermission(Unknown Source)

at java.lang.SecurityManager.checkPermission(Unknown Source)

at java.lang.SecurityManager.checkPropertyAccess(Unknown Source)

at java.lang.System.getProperty(Unknown Source)

at org.jboss.util.NestedThrowable$Util.getBoolean(NestedThrowable.java:123)

at org.jboss.util.NestedThrowable.(NestedThrowable.java:39)

... 9 more


And the server app gets this

EventJournal.writeMsg org.jboss.mq.SpyJMSException: Cannot send a message to the JMS server; - nested throwable: (java.rmi.Unmarshal
Exception: Error unmarshaling return header; nested exception is:
java.io.EOFException)
org.jboss.mq.SpyJMSException: Cannot send a message to the JMS server; - nested throwable: (java.net.SocketException: Connection res
et)
at org.jboss.mq.Connection.sendToServer(Connection.java:1127)
at org.jboss.mq.SpySession.sendMessage(SpySession.java:654)
at org.jboss.mq.SpyTopicPublisher.internalPublish(SpyTopicPublisher.java:120)
at org.jboss.mq.SpyTopicPublisher.publish(SpyTopicPublisher.java:84)
at com.uss.irv.cr84.sys.jms.TopicBroadcaster.publish(TopicBroadcaster.java:113)
at com.uss.irv.cr84.sys.jms.TopicBroadcaster.send(TopicBroadcaster.java:76)
at com.uss.irv.cr84.rollshop.grind.GrinderBroadcaster.broadcast(GrinderBroadcaster.java:40)
at com.uss.irv.cr84.rollshop.grind.Grinder.insRoughness(Grinder.java:1081)
at com.uss.irv.cr84.rollshop.grind.GrinderQMsgHandler.processMsg(GrinderQMsgHandler.java:198)
at com.uss.irv.cr84.rollshop.grind.GrinderQMsgReader.inMsg(GrinderQMsgReader.java:52)
at com.uss.irv.cr84.rollshop.grind.JRsGrinder.onMessage(JRsGrinder.java:77)
at org.jboss.mq.SpyMessageConsumer.run(SpyMessageConsumer.java:526)
at java.lang.Thread.run(Thread.java:536)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2316)
at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2383)
at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2455)
at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2604)
at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
at org.jboss.mq.il.oil.OILServerIL.waitAnswer(OILServerIL.java:589)
at org.jboss.mq.il.oil.OILServerIL.addMessage(OILServerIL.java:219)
at org.jboss.mq.Connection.sendToServer(Connection.java:1119)
... 12 more


Any ideas? I am trying to play around with some of the configuration settings to help. I upped the max memory and got it work longer before it dies.

[cholliday]

Never mind my MaxMemoryMark change. I don't believe that has any effect because the current memory isn't even close to that.

[cholliday]

I started with the default 500 HighWater Mark and 600 MaxMemoryMark and died with an OutOfMemory error. When I upped MaxMemoryMark it lasted about 4 times longer but still died with the OutOfMemory error. I just bumped it down to 100 & 200 for the HighWaterMark & MaxMemoryMark respectfully and I get this crash dump (file attached). My CurrentMemoryUsage though stays at about 15 so I don't understand how changing these numbers effect my application but it seems to. I guess I don't understand what CurrentMemoryUsage is. I can't find anything in the documentation I bought that helps me much. Any help will be much appreciated.

[cholliday]

I am not having the OutOfMemory issues anymore because I went away from using persistance totally and I went away from any kind acknowledgement from the Queue that has the most throughput. Now I keep getting bug check dumps. I guess I should talk about this in a new thread though.