4 Replies Latest reply on May 27, 2010 8:21 AM by mnenchev

Jboss out of memory problem

mnenchev May 26, 2010 4:53 AM

Hi, i have serious problem with jboss out of memory. Please see the log and advice me how to fix it.

<code>

2010-05-24 05:45:42,649 ERROR [org.jgroups.protocols.TCP] (OOB-11043,10.35.12.2:7900) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
    at java.util.HashMap.entrySet0(HashMap.java:945)
    at java.util.HashMap.entrySet(HashMap.java:940)
    at java.util.concurrent.ConcurrentHashMap.putAll(ConcurrentHashMap.java:908)
    at org.jgroups.util.Digest.readFrom(Digest.java:379)
    at org.jgroups.util.Util.readStreamable(Util.java:645)
    at org.jgroups.protocols.pbcast.STABLE$StableHeader.readFrom(STABLE.java:771)
    at org.jgroups.Message.readHeader(Message.java:779)
    at org.jgroups.Message.readFrom(Message.java:673)
    at org.jgroups.protocols.TP.readMessage(TP.java:1514)
    at org.jgroups.protocols.TP.access$700(TP.java:49)
    at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1816)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
2010-05-24 05:46:10,264 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-1) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
    at java.lang.AbstractStringBuilder.<init>(AbstractStringBuilder.java:45)
    at java.lang.StringBuilder.<init>(StringBuilder.java:68)
    at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.getPathName(AbstractVirtualFileHandler.java:219)
    at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.readLocalPathName(AbstractVirtualFileHandler.java:260)
    at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.getLocalPathName(AbstractVirtualFileHandler.java:244)
    at org.jboss.virtual.plugins.context.zip.ZipEntryContext.initEntries(ZipEntryContext.java:583)
    at org.jboss.virtual.plugins.context.zip.ZipEntryContext.ensureEntries(ZipEntryContext.java:619)
    at org.jboss.virtual.plugins.context.zip.ZipEntryContext.checkIfModified(ZipEntryContext.java:773)
    at org.jboss.virtual.plugins.context.zip.ZipEntryContext.getChild(ZipEntryContext.java:817)
    at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.createChildHandler(ZipEntryHandler.java:191)
    at org.jboss.virtual.plugins.context.AbstractVirtualFileHandler.structuredFindChild(AbstractVirtualFileHandler.java:684)
    at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.getChild(ZipEntryHandler.java:165)
    at org.jboss.virtual.plugins.context.DelegatingHandler.getChild(DelegatingHandler.java:107)
    at org.jboss.virtual.VirtualFile.findChild(VirtualFile.java:457)
    at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.resolveVirtualFile(VirtualFileURLConnection.java:106)
    at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getVirtualFile(VirtualFileURLConnection.java:118)
    at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:93)
    at java.net.URL.openStream(URL.java:1010)
    at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1170)
    at org.jboss.ws.core.utils.JBossWSEntityResolver$1.run(JBossWSEntityResolver.java:80)
    at org.jboss.ws.core.utils.JBossWSEntityResolver$1.run(JBossWSEntityResolver.java:78)
    at java.security.AccessController.doPrivileged(Native Method)
    at org.jboss.ws.core.utils.JBossWSEntityResolver.loadEntitiesMappingFromClasspath(JBossWSEntityResolver.java:76)
    at org.jboss.ws.core.utils.JBossWSEntityResolver.<init>(JBossWSEntityResolver.java:59)
    at org.jboss.ws.core.utils.JBossWSEntityResolver.<init>(JBossWSEntityResolver.java:51)
    at sun.reflect.GeneratedConstructorAccessor365.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
    at java.lang.Class.newInstance0(Class.java:355)
    at java.lang.Class.newInstance(Class.java:308)
    at org.jboss.wsf.common.DOMUtils$1.setEntityResolver(DOMUtils.java:121)
    at org.jboss.wsf.common.DOMUtils$1.initialValue(DOMUtils.java:101)
2010-05-24 08:33:58,387 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-1) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:36:12,916 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-1) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:38:57,558 ERROR [STDERR] (ZipFile Lock Reaper) Exception in thread "ZipFile Lock Reaper"
2010-05-24 08:38:57,558 ERROR [STDERR] (ZipFile Lock Reaper) java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:38:57,558 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-6) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:40:05,175 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-10) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:41:21,031 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-2) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:43:49,700 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-10) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:46:10,133 ERROR [org.jgroups] (OOB-11369,10.35.12.2:7900) uncaught exception in Thread[OOB-11369,10.35.12.2:7900,5,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:46:18,313 ERROR [STDERR] (DefaultQuartzScheduler_QuartzSchedulerThread) Exception in thread "DefaultQuartzScheduler_QuartzSchedulerThread"
2010-05-24 08:46:18,313 ERROR [org.jgroups.protocols.TCP] (OOB-11370,10.35.12.2:7900) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:46:22,306 ERROR [STDERR] (DefaultQuartzScheduler_QuartzSchedulerThread) java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:50:45,047 ERROR [STDERR] (AOPListner) Exception in thread "AOPListner"
2010-05-24 08:50:45,047 ERROR [STDERR] (AOPListner) java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:51:37,051 ERROR [org.jgroups.util.TimeScheduler] (Timer-2,10.35.12.2:7900) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@971e07
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:51:37,051 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-1,10.35.12.2:7900) failure passing message down
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:51:37,051 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-3,10.35.12.2:7900) failure passing message down
java.lang.reflect.UndeclaredThrowableException
    at org.apache.commons.logging.impl.Log4jProxy.log(Log4jProxy.java:309)
    at org.apache.commons.logging.impl.Log4jProxy.error(Log4jProxy.java:283)
    at org.apache.commons.logging.impl.Log4JLogger.error(Log4JLogger.java:208)
    at org.jgroups.protocols.TP.down(TP.java:1205)
    at org.jgroups.protocols.TP$ProtocolAdapter.down(TP.java:2316)
    at org.jgroups.protocols.Discovery.down(Discovery.java:374)
    at org.jgroups.protocols.MERGE2.down(MERGE2.java:175)
    at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:362)
    at org.jgroups.protocols.FD.down(FD.java:315)
    at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:95)
    at org.jgroups.protocols.BARRIER.down(BARRIER.java:107)
    at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:810)
    at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:612)
    at org.jgroups.protocols.UNICAST.down(UNICAST.java:462)
    at org.jgroups.protocols.pbcast.STABLE$1.run(STABLE.java:644)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:51:37,054 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-1) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 08:53:56,807 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-2) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:27:05,114 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-2) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:32:00,768 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-2) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:52:40,557 ERROR [STDERR] (main-FastReceiver) Exception in thread "main-FastReceiver"
2010-05-24 09:52:40,557 ERROR [STDERR] (main-FastReceiver) java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:52:40,557 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-11,10.35.12.2:36017) failure passing message down
java.lang.reflect.UndeclaredThrowableException
    at org.apache.commons.logging.impl.Log4jProxy.log(Log4jProxy.java:309)
    at org.apache.commons.logging.impl.Log4jProxy.error(Log4jProxy.java:283)
    at org.apache.commons.logging.impl.Log4JLogger.error(Log4JLogger.java:208)
    at org.jgroups.protocols.TP.down(TP.java:1205)
    at org.jgroups.protocols.TP$ProtocolAdapter.down(TP.java:2316)
    at org.jgroups.protocols.Discovery.down(Discovery.java:374)
    at org.jgroups.protocols.MERGE2.down(MERGE2.java:175)
    at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:362)
    at org.jgroups.protocols.FD.down(FD.java:315)
    at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:95)
    at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:810)
    at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:612)
    at org.jgroups.protocols.UNICAST.down(UNICAST.java:462)
    at org.jgroups.protocols.pbcast.STABLE$StabilitySendTask.run(STABLE.java:855)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:52:40,558 WARN [org.jboss.system.server.profileservice.hotdeploy.HDScanner] (HDScanner) Scan failed
java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:52:46,653 ERROR [STDERR] (SubscriptionWatchDog) Exception in thread "SubscriptionWatchDog"
2010-05-24 09:52:46,653 ERROR [STDERR] (SubscriptionWatchDog) java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:53:12,414 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-1) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:53:12,415 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-2) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:53:12,416 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-10) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 09:58:12,185 ERROR [org.jgroups] (OOB--28175,10.35.12.2:36017) uncaught exception in Thread[OOB--28175,10.35.12.2:36017,5,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.OutOfMemoryError: Java heap space
2010-05-24 11:18:01,497 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-2) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 11:19:40,988 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-10) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 13:47:27,892 ERROR [org.jgroups.protocols.UDP] (Timer-6,10.35.12.2:36017) failed sending message to null (162 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 13:48:03,810 ERROR [org.jgroups.util.TimeScheduler] (Timer-1,10.35.12.2:36017) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@5b54e6
java.lang.OutOfMemoryError: Java heap space
2010-05-24 13:47:55,635 ERROR [STDERR] (RMI TCP Connection(idle)) Exception in thread "RMI TCP Connection(idle)"
2010-05-24 13:48:23,481 ERROR [org.jgroups.util.TimeScheduler] (Timer-1,10.35.12.2:7900) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@971e07
java.lang.OutOfMemoryError: Java heap space
2010-05-24 13:57:44,626 ERROR [STDERR] (RMI TCP Connection(idle)) java.lang.OutOfMemoryError: Java heap space
2010-05-24 13:58:04,391 ERROR [STDERR] (RMI RenewClean-[10.35.12.2:1101]) Exception in thread "RMI RenewClean-[10.35.12.2:1101]"
2010-05-24 14:00:58,286 ERROR [STDERR] (RMI RenewClean-[10.35.12.2:1101]) java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:01:02,788 ERROR [STDERR] (Thread-19) Exception in thread "Thread-19"
2010-05-24 14:01:02,788 ERROR [STDERR] (Thread-19) java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:11:26,037 ERROR [org.jgroups] (OOB--27597,10.35.12.2:36017) uncaught exception in Thread[OOB--27597,10.35.12.2:36017,5,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:37:32,020 ERROR [org.jgroups.protocols.UDP] (Timer-4,10.35.12.2:36017) failed sending message to null (162 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:40:17,091 ERROR [org.jgroups.protocols.UDP] (OOB-1,PojoCacheCluster,10.35.12.2:36454) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:40:17,091 ERROR [org.jgroups.util.TimeScheduler] (Timer-1,PojoCacheCluster,10.35.12.2:36454) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@102eca7
java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:39:10,051 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-10,10.35.12.2:36017) failure passing message down
java.lang.reflect.UndeclaredThrowableException
    at org.apache.commons.logging.impl.Log4jProxy.log(Log4jProxy.java:309)
    at org.apache.commons.logging.impl.Log4jProxy.error(Log4jProxy.java:283)
    at org.apache.commons.logging.impl.Log4JLogger.error(Log4JLogger.java:208)
    at org.jgroups.protocols.TP.down(TP.java:1205)
    at org.jgroups.protocols.TP$ProtocolAdapter.down(TP.java:2316)
    at org.jgroups.protocols.Discovery.down(Discovery.java:374)
    at org.jgroups.protocols.MERGE2.down(MERGE2.java:175)
    at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:362)
    at org.jgroups.protocols.FD.down(FD.java:315)
    at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:95)
    at org.jgroups.protocols.BARRIER.down(BARRIER.java:107)
    at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:810)
    at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:612)
    at org.jgroups.protocols.UNICAST.down(UNICAST.java:462)
    at org.jgroups.protocols.pbcast.STABLE$1.run(STABLE.java:644)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:38:11,297 WARN [org.jboss.system.server.profileservice.hotdeploy.HDScanner] (HDScanner) Scan failed
java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:40:24,878 ERROR [STDERR] (main-FastReceiver) Exception in thread "main-FastReceiver"
2010-05-24 14:40:24,878 ERROR [STDERR] (main-FastReceiver) java.lang.OutOfMemoryError: Java heap space
2010-05-24 14:40:24,878 ERROR [org.jgroups.protocols.UDP] (Timer-1,10.35.12.2:36017) failed sending message to null (159 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:01:05,206 ERROR [org.jgroups] (OOB--27554,10.35.12.2:36017) uncaught exception in Thread[OOB--27554,10.35.12.2:36017,5,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:22:13,935 ERROR [org.jgroups.protocols.UDP] (Timer-3,PojoCacheCluster,10.35.12.2:36454) failed sending message to null (115 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:23:40,386 WARN [org.jboss.system.server.profileservice.hotdeploy.HDScanner] (HDScanner) Scan failed
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:22:57,050 ERROR [org.jgroups.protocols.TCP] (Timer-4,10.35.12.2:7900) failed handling data from 10.35.12.2:7900
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:23:56,089 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-3,10.35.12.2:7900) failure passing message down
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:23:56,089 ERROR [org.apache.catalina.core.ContainerBase] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Exception invoking periodic operation:
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:25:23,135 ERROR [org.jgroups.util.TimeScheduler] (Timer-2,10.35.12.2:7900) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@971e07
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:40:25,476 ERROR [STDERR] (main-FastReceiver) Exception in thread "main-FastReceiver"
2010-05-24 15:40:25,477 ERROR [STDERR] (main-FastReceiver) java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:40:29,681 WARN [org.jboss.system.server.profileservice.hotdeploy.HDScanner] (HDScanner) Scan failed
java.lang.OutOfMemoryError: Java heap space
2010-05-24 15:40:29,681 ERROR [org.jgroups.protocols.UDP] (OOB-1,PojoCacheCluster,10.35.12.2:36454) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:15:02,031 WARN [org.jboss.system.server.profileservice.hotdeploy.HDScanner] (HDScanner) Scan failed
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:18:53,850 ERROR [org.jgroups.protocols.TCP] (Timer-3,10.35.12.2:7900) failed sending message to null (157 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:23:06,322 ERROR [org.jgroups.protocols.UDP] (Timer-2,10.35.12.2:36017) failed sending message to null (162 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:45:29,456 ERROR [STDERR] (MaintThread) Exception in thread "MaintThread"
2010-05-24 16:51:13,905 ERROR [org.jgroups.protocols.UDP] (OOB-2,PojoCacheCluster,10.35.12.2:36454) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:50:20,769 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-1,10.35.12.2:36017) failure passing message down
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:50:05,042 ERROR [org.jgroups.protocols.UDP] (Timer-6,10.35.12.2:36017) failed sending message to null (159 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:52:41,962 ERROR [STDERR] (MaintThread) java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:51:50,978 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failure in multicast receive()
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:51:27,529 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/clairews].[AccountService]] (http-10.35.12.2-8080-10) Servlet.service() for servlet AccountService threw exception
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:01:36,224 ERROR [STDERR] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Exception in thread "ContainerBackgroundProcessor[StandardEngine[jboss.web]]"
2010-05-24 17:01:08,522 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-10,10.35.12.2:36017) failure passing message down
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:01:04,625 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-12,10.35.12.2:36017) failure passing message down
java.lang.reflect.UndeclaredThrowableException
Caused by: java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:00:01,752 ERROR [org.jgroups.protocols.UDP] (Timer-3,10.35.12.2:36017) failed sending message to null (166 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:59:38,208 ERROR [org.jgroups.util.TimeScheduler] (Timer-1,10.35.12.2:36017) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@16243cc
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:59:22,557 ERROR [org.jgroups.util.TimeScheduler] (Timer-7,10.35.12.2:36017) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@4ae89c
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:56:02,315 ERROR [org.jgroups.protocols.UDP] (Timer-8,10.35.12.2:36017) failed sending message to null (149 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:55:46,480 ERROR [org.jgroups.util.TimeScheduler] (Timer-2,10.35.12.2:7900) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@971e07
java.lang.OutOfMemoryError: Java heap space
2010-05-24 16:54:55,295 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-9,10.35.12.2:36017) failure passing message down
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:04:06,370 ERROR [org.jgroups] (OOB--27477,10.35.12.2:36017) uncaught exception in Thread[OOB--27477,10.35.12.2:36017,5,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:03:38,521 ERROR [STDERR] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:03:34,589 ERROR [org.apache.catalina.connector.CoyoteAdapter] (http-10.35.12.2-8080-10) An exception or error occurred in the container during the request processing
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:03:10,945 ERROR [org.jgroups] (OOB--27476,10.35.12.2:36017) uncaught exception in Thread[OOB--27476,10.35.12.2:36017,5,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:02:39,344 WARN [org.jgroups.protocols.pbcast.NAKACK] (Timer-6,10.35.12.2:36017) failure passing message down
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:05:21,532 ERROR [org.apache.coyote.http11.Http11Processor] (http-10.35.12.2-8080-10) Error finishing response
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:05:25,520 ERROR [org.jgroups] (OOB--27481,10.35.12.2:36017) uncaught exception in Thread[OOB--27481,10.35.12.2:36017,5,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:09:53,916 ERROR [org.jgroups.protocols.UDP] (OOB--27444,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:09:06,621 ERROR [org.jgroups.protocols.UDP] (OOB--27452,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:10:53,122 ERROR [org.jgroups.protocols.UDP] (OOB--27445,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:10:49,204 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failed handling data from 10.35.12.2:45688
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:10:23,686 ERROR [org.jgroups.protocols.UDP] (OOB--27447,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:15:59,459 ERROR [org.jgroups.protocols.UDP] (Timer-13,10.35.12.2:36017) failed sending message to null (149 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:14:45,996 ERROR [org.jgroups.protocols.UDP] (OOB--27441,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:14:20,096 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failure in multicast receive()
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:13:32,725 ERROR [org.jgroups.protocols.UDP] (Timer-8,10.35.12.2:36017) failed sending message to null (149 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:13:04,946 ERROR [org.jgroups.util.TimeScheduler] (Timer-5,10.35.12.2:7900) failed running task org.jgroups.protocols.pbcast.STABLE$StableTask@971e07
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:17:03,015 ERROR [org.jgroups.protocols.UDP] (Timer-12,10.35.12.2:36017) failed sending message to null (159 bytes)
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:18:45,255 ERROR [org.jgroups.protocols.UDP] (OOB--27434,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:19:17,081 ERROR [org.jgroups.protocols.UDP] (OOB--27438,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:19:52,439 ERROR [org.jgroups.protocols.UDP] (OOB--27437,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:21:23,758 ERROR [org.jgroups.protocols.UDP] (OOB--27438,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:23:06,921 ERROR [org.jgroups.protocols.UDP] (OOB--27433,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:22:43,100 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failed handling data from 10.35.12.2:45688
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:24:24,145 ERROR [org.jgroups.protocols.UDP] (OOB--27431,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:25:51,226 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failure in multicast receive()
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:27:55,549 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failure in multicast receive()
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:28:23,154 ERROR [org.jgroups] (Timer-5,10.35.12.2:7900) uncaught exception in Thread[Timer-5,10.35.12.2:7900,5,JGroups] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:31:24,198 ERROR [org.jgroups.protocols.UDP] (OOB--27413,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:34:17,547 ERROR [org.jgroups.protocols.UDP] (OOB--27410,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:34:33,301 ERROR [org.jgroups.protocols.UDP] (OOB--27409,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:36:31,858 ERROR [org.jgroups.protocols.UDP] (OOB--27400,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:36:55,345 ERROR [org.jgroups.protocols.UDP] (OOB--27408,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:42:10,395 ERROR [org.jgroups] (OOB--27402,10.35.12.2:36017) uncaught exception in Thread[OOB--27402,10.35.12.2:36017,10,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.reflect.UndeclaredThrowableException
    at org.apache.commons.logging.impl.Log4jProxy.log(Log4jProxy.java:309)
    at org.apache.commons.logging.impl.Log4jProxy.error(Log4jProxy.java:283)
    at org.apache.commons.logging.impl.Log4JLogger.error(Log4JLogger.java:208)
    at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1822)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:42:34,103 ERROR [org.jgroups] (OOB--27401,10.35.12.2:36017) uncaught exception in Thread[OOB--27401,10.35.12.2:36017,10,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.reflect.UndeclaredThrowableException
Caused by: java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:47:02,382 ERROR [org.jgroups.protocols.UDP] (OOB--27383,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:48:09,072 ERROR [org.jgroups.protocols.UDP] (OOB--27378,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:50:00,694 ERROR [org.jgroups.protocols.UDP] (OOB--27381,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:52:12,690 ERROR [org.jgroups.protocols.UDP] (OOB--27373,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:52:20,540 ERROR [org.jgroups.protocols.UDP] (OOB--27375,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:52:36,204 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failed handling data from 10.35.12.2:45688
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:52:44,023 ERROR [org.jgroups] (OOB--27374,10.35.12.2:36017) uncaught exception in Thread[OOB--27374,10.35.12.2:36017,10,Thread Pools] (thread group=org.jgroups.util.Util$1[name=JGroups,maxpri=10] )
java.lang.reflect.UndeclaredThrowableException
    at org.apache.commons.logging.impl.Log4jProxy.log(Log4jProxy.java:309)
    at org.apache.commons.logging.impl.Log4jProxy.error(Log4jProxy.java:283)
    at org.apache.commons.logging.impl.Log4JLogger.error(Log4JLogger.java:208)
    at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1822)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:58:10,635 ERROR [org.jgroups.protocols.UDP] (OOB--27343,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:58:24,413 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failed handling data from 10.35.12.2:45688
java.lang.OutOfMemoryError: Java heap space
2010-05-24 17:58:32,252 ERROR [org.jgroups.protocols.UDP] (OOB--27342,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:04:04,192 ERROR [org.jgroups.protocols.UDP] (OOB--27303,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:04:19,853 ERROR [org.jgroups.protocols.UDP] (OOB--27301,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:08:17,667 ERROR [org.apache.coyote.ajp.AjpProtocol] (ajp-10.35.12.2-8009-119) Error reading request, ignored
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:08:21,572 ERROR [STDERR] (ajp-10.35.12.2-8009-119) Exception in thread "ajp-10.35.12.2-8009-119"
2010-05-24 18:08:23,539 ERROR [STDERR] (ajp-10.35.12.2-8009-119) java.lang.NullPointerException
2010-05-24 18:08:23,540 ERROR [STDERR] (ajp-10.35.12.2-8009-119)     at java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:190)
2010-05-24 18:08:31,350 ERROR [STDERR] (ajp-10.35.12.2-8009-119)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:340)
2010-05-24 18:08:35,240 ERROR [STDERR] (ajp-10.35.12.2-8009-119)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:357)
2010-05-24 18:08:39,150 ERROR [STDERR] (ajp-10.35.12.2-8009-119)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:411)
2010-05-24 18:08:43,037 ERROR [STDERR] (ajp-10.35.12.2-8009-119)     at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
2010-05-24 18:08:46,923 ERROR [STDERR] (ajp-10.35.12.2-8009-119)     at java.lang.Thread.run(Thread.java:619)
2010-05-24 18:14:06,936 ERROR [org.jgroups.protocols.UDP] (OOB--27245,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:14:38,295 ERROR [org.jgroups.protocols.UDP] (OOB--27247,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:14:53,904 ERROR [org.jgroups.protocols.UDP] (OOB--27244,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:15:23,779 ERROR [org.jgroups.protocols.UDP] (OOB--27253,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:15:35,530 ERROR [org.jgroups.protocols.UDP] (OOB--27243,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:19:47,259 ERROR [org.jgroups.protocols.UDP] (OOB--27222,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:19:43,334 ERROR [org.jgroups.protocols.UDP] (OOB--27225,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:19:23,525 ERROR [org.jgroups.protocols.UDP] (OOB--27224,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:18:39,965 ERROR [org.jgroups.protocols.UDP] (OOB--27227,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:20:26,717 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failure in multicast receive()
java.lang.reflect.UndeclaredThrowableException
    at org.apache.commons.logging.impl.Log4jProxy.log(Log4jProxy.java:309)
    at org.apache.commons.logging.impl.Log4jProxy.error(Log4jProxy.java:283)
    at org.apache.commons.logging.impl.Log4JLogger.error(Log4JLogger.java:208)
    at org.jgroups.protocols.TP.receive(TP.java:1352)
    at org.jgroups.protocols.UDP.run(UDP.java:277)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:20:03,093 ERROR [org.jgroups.protocols.UDP] (OOB--27223,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:22:13,444 ERROR [org.jgroups.protocols.UDP] (OOB--27221,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:22:09,512 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failed handling data from 10.35.12.2:45688
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:23:31,990 ERROR [org.jgroups.protocols.UDP] (OOB--27221,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:24:11,474 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failed handling data from 10.35.12.2:45688
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:24:37,003 ERROR [org.jgroups.protocols.UDP] (OOB--27217,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:26:45,395 ERROR [org.jgroups.protocols.UDP] (OOB--27215,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:26:21,830 ERROR [org.jgroups.protocols.UDP] (OOB--27217,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:25:58,054 ERROR [org.jgroups.protocols.UDP] (OOB--27212,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:25:40,400 ERROR [org.jgroups.protocols.UDP] (OOB--27214,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:28:09,591 ERROR [org.jgroups.protocols.UDP] (UDP mcast,10.35.12.2:36017) failure in multicast receive()
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:32:00,601 ERROR [org.jgroups.protocols.UDP] (OOB--27191,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:32:49,716 ERROR [org.jgroups.protocols.UDP] (OOB--27192,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:33:33,170 ERROR [org.jgroups.protocols.UDP] (OOB--27190,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:33:17,408 ERROR [org.jgroups.protocols.UDP] (OOB--27191,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:34:28,134 ERROR [org.jgroups.protocols.UDP] (OOB--27188,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:34:47,695 ERROR [org.jgroups.protocols.UDP] (OOB--27189,10.35.12.2:36017) failed handling incoming message
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:54:02,004 ERROR [org.apache.coyote.ajp.AjpProtocol] (ajp-10.35.12.2-8009-120) Error reading request, ignored
java.lang.OutOfMemoryError: Java heap space
2010-05-24 18:54:05,926 ERROR [STDERR] (ajp-10.35.12.2-8009-120) Exception in thread "ajp-10.35.12.2-8009-120"
2010-05-24 18:54:05,926 ERROR [STDERR] (ajp-10.35.12.2-8009-120) java.lang.NullPointerException
2010-05-24 18:54:09,819 ERROR [STDERR] (ajp-10.35.12.2-8009-120)     at java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:190)
2010-05-24 18:54:15,695 ERROR [STDERR] (ajp-10.35.12.2-8009-120)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:340)
2010-05-24 18:54:17,642 ERROR [STDERR] (ajp-10.35.12.2-8009-120)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:357)
2010-05-24 18:54:25,463 ERROR [STDERR] (ajp-10.35.12.2-8009-120)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:411)
2010-05-24 18:54:29,374 ERROR [STDERR] (ajp-10.35.12.2-8009-120)     at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
2010-05-24 18:54:29,374 ERROR [STDERR] (ajp-10.35.12.2-8009-120)     at java.lang.Thread.run(Thread.java:619)
2010-05-24 19:43:20,437 ERROR [org.apache.coyote.ajp.AjpProtocol] (ajp-10.35.12.2-8009-121) Error reading request, ignored
java.lang.OutOfMemoryError: Java heap space
2010-05-24 19:43:24,339 ERROR [STDERR] (ajp-10.35.12.2-8009-121) Exception in thread "ajp-10.35.12.2-8009-121"
2010-05-24 19:43:24,339 ERROR [STDERR] (ajp-10.35.12.2-8009-121) java.lang.NullPointerException
2010-05-24 19:43:28,273 ERROR [STDERR] (ajp-10.35.12.2-8009-121)     at java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:190)
2010-05-24 19:43:34,183 ERROR [STDERR] (ajp-10.35.12.2-8009-121)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:340)
2010-05-24 19:43:36,140 ERROR [STDERR] (ajp-10.35.12.2-8009-121)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:357)
2010-05-24 19:43:43,994 ERROR [STDERR] (ajp-10.35.12.2-8009-121)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:411)
2010-05-24 19:43:47,909 ERROR [STDERR] (ajp-10.35.12.2-8009-121)     at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
2010-05-24 19:43:47,909 ERROR [STDERR] (ajp-10.35.12.2-8009-121)     at java.lang.Thread.run(Thread.java:619)
2010-05-24 20:27:58,265 ERROR [org.apache.coyote.ajp.AjpProtocol] (ajp-10.35.12.2-8009-122) Error reading request, ignored
java.lang.OutOfMemoryError: Java heap space
2010-05-24 20:28:02,352 ERROR [STDERR] (ajp-10.35.12.2-8009-122) Exception in thread "ajp-10.35.12.2-8009-122"
2010-05-24 20:28:02,352 ERROR [STDERR] (ajp-10.35.12.2-8009-122) java.lang.NullPointerException
2010-05-24 20:28:06,295 ERROR [STDERR] (ajp-10.35.12.2-8009-122)     at java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:190)
2010-05-24 20:28:12,153 ERROR [STDERR] (ajp-10.35.12.2-8009-122)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:340)
2010-05-24 20:28:14,091 ERROR [STDERR] (ajp-10.35.12.2-8009-122)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:357)
2010-05-24 20:28:21,995 ERROR [STDERR] (ajp-10.35.12.2-8009-122)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:411)
2010-05-24 20:28:25,916 ERROR [STDERR] (ajp-10.35.12.2-8009-122)     at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
2010-05-24 20:28:25,916 ERROR [STDERR] (ajp-10.35.12.2-8009-122)     at java.lang.Thread.run(Thread.java:619)
2010-05-24 21:05:46,185 ERROR [org.apache.coyote.ajp.AjpProtocol] (ajp-10.35.12.2-8009-123) Error reading request, ignored
java.lang.OutOfMemoryError: Java heap space
2010-05-24 21:05:50,081 ERROR [STDERR] (ajp-10.35.12.2-8009-123) Exception in thread "ajp-10.35.12.2-8009-123"
2010-05-24 21:05:50,081 ERROR [STDERR] (ajp-10.35.12.2-8009-123) java.lang.NullPointerException
2010-05-24 21:05:53,955 ERROR [STDERR] (ajp-10.35.12.2-8009-123)     at java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:190)
2010-05-24 21:05:59,773 ERROR [STDERR] (ajp-10.35.12.2-8009-123)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:340)
2010-05-24 21:06:01,741 ERROR [STDERR] (ajp-10.35.12.2-8009-123)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:357)
2010-05-24 21:06:09,585 ERROR [STDERR] (ajp-10.35.12.2-8009-123)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:411)
2010-05-24 21:06:13,509 ERROR [STDERR] (ajp-10.35.12.2-8009-123)     at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
2010-05-24 21:06:13,509 ERROR [STDERR] (ajp-10.35.12.2-8009-123)     at java.lang.Thread.run(Thread.java:619)
2010-05-24 22:48:59,264 ERROR [org.apache.coyote.ajp.AjpProtocol] (ajp-10.35.12.2-8009-124) Error reading request, ignored
java.lang.OutOfMemoryError: Java heap space
2010-05-24 22:49:03,360 ERROR [STDERR] (ajp-10.35.12.2-8009-124) Exception in thread "ajp-10.35.12.2-8009-124"
2010-05-24 22:49:03,360 ERROR [STDERR] (ajp-10.35.12.2-8009-124) java.lang.NullPointerException
2010-05-24 22:49:05,342 ERROR [STDERR] (ajp-10.35.12.2-8009-124)     at java.util.concurrent.ConcurrentLinkedQueue.offer(ConcurrentLinkedQueue.java:190)
2010-05-24 22:49:09,224 ERROR [STDERR] (ajp-10.35.12.2-8009-124)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:340)
2010-05-24 22:49:13,150 ERROR [STDERR] (ajp-10.35.12.2-8009-124)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler$1.offer(AjpProtocol.java:357)
2010-05-24 22:49:20,948 ERROR [STDERR] (ajp-10.35.12.2-8009-124)     at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:411)
2010-05-24 22:49:20,948 ERROR [STDERR] (ajp-10.35.12.2-8009-124)     at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
2010-05-24 22:49:26,821 ERROR [STDERR] (ajp-10.35.12.2-8009-124)     at java.lang.Thread.run(Thread.java:619)

</code>

1. Re: Jboss out of memory problem

saltnlight5 May 26, 2010 7:48 AM (in response to mnenchev)

Have you try increasing the Java heap memory?

If you are using JBoss5, edit the bin/run.conf(.bat) under the JAVA_OPTS variable and change the -Xmx value.

*Read http://community.jboss.org/wiki/OutOfMemoryExceptions
Actions
2. Re: Jboss out of memory problem

mnenchev May 26, 2010 8:29 AM (in response to saltnlight5)

Hi, thanks. I set this to 2048, it was 512. How can i effectively monitor the memory, and if there are memory leaks. I use alot JMS - jboss messaging, websphere&activemq rars. May be they course some memory leak?
Actions
3. Re: Jboss out of memory problem

peterj May 26, 2010 11:28 AM (in response to mnenchev)

You can add -XX:+HeapDumpOnOutOfMemoryError to the JAVA_OPTS in the run script - this option will dump the heap on an OOME. You can then use VisualVM (part of JDK 6, or download for JDK 5) to analyze the heap dump, that should give you some idea as to what is causing them emory leak.

You can also use VisualVM and JConsole to generated a heap dump at any time.

But then it might not be a memeory leak, some apps just need more heap space, especially if they get real busy. So you might just need more that a 512MB heap.
Actions
4. Re: Jboss out of memory problem

mnenchev May 27, 2010 8:21 AM (in response to peterj)

You are right. My app is busy all the time.
Actions

Go to original post