0 Replies Latest reply on Aug 2, 2010 12:55 PM by samavedulark

    java.lang.OutOfMemoryError: Java heap space  <[org.jgroups.protocols.UDP] >

    samavedulark

      Hi,

       

      I was running an Loadtest in jboss eap 5.0 .It ran well for 3 hours and then i found error in server.log.

      Below is the error.

       

       

      2010-08-02 16:21:04,235 WARN  [org.jgroups.protocols.UDP] (JBoss System Threads(1)-3) failed to join /224.0.75.75:7500 on net4: java.net.SocketException: Unrecognized Windows Sockets error: 0: no Inet4Address associated with interface
      2010-08-02 16:21:04,235 WARN  [org.jgroups.protocols.UDP] (JBoss System Threads(1)-3) failed to join /224.0.75.75:7500 on net6: java.net.SocketException: Unrecognized Windows Sockets error: 0: no Inet4Address associated with interface
      2010-08-02 16:21:04,235 INFO  [STDOUT] (JBoss System Threads(1)-3)
      ---------------------------------------------------------
      GMS: address is 3.204.35.154:59962 (cluster=CDI)
      ---------------------------------------------------------
      2010-08-02 16:21:04,578 INFO  [STDOUT] (main)
      ---------------------------------------------------------
      GMS: address is 3.204.35.154:59962 (cluster=CDI-HAPartitionCache)
      ---------------------------------------------------------
      2010-08-02 16:21:13,610 WARN  [org.jgroups.protocols.UDP] (main) failed to join /224.0.75.75:7500 on net4: java.net.SocketException: Unrecognized Windows Sockets error: 0: no Inet4Address associated with interface
      2010-08-02 16:21:13,610 WARN  [org.jgroups.protocols.UDP] (main) failed to join /224.0.75.75:7500 on net6: java.net.SocketException: Unrecognized Windows Sockets error: 0: no Inet4Address associated with interface
      2010-08-02 16:21:13,626 INFO  [STDOUT] (main)
      ---------------------------------------------------------
      GMS: address is 3.204.35.154:59964 (cluster=JBossCache-Cluster)
      ---------------------------------------------------------
      2010-08-02 16:21:16,325 INFO  [org.apache.catalina.core.AprLifecycleListener] (main) The Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\GeHealthcare\Centricity\DeviceDataProcessor\jdk1.6.0_20\bin;.;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\GEHEAL~1\CENTRI~1\DEVICE~1\oraclexe\app\oracle\product\10.2.0\server\bin;c:\gehc-it\ccg\quovadx\qdx5.7MB\integrator\sbin;C:\Program Files (x86)\Java\jre1.5.0_11\bin;C:\Program Files\HP\NCU;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;D:\QS\CDIAdapter;D:\QS\CDIAdapter;C:\Program Files (x86)\Java\jre1.5.0_11\bin;D:\QS\CDIAdapter;C:\Program Files (x86)\Java\jre1.5.0_11\bin;D:\QS\CDIAdapter;C:\Program Files (x86)\Java\jre1.5.0_11\bin;c:\gehc-it\_jvm\bin;c:\gehc-it\ccg\backup\bin;c:\gehc-it\ccg\bin;C:\GeHealthcare\Centricity\DeviceDataProcessor\instantclient_10_2;D:\QS\TOOLS;D:\QS\EXE;
      2010-08-02 16:21:16,403 INFO  [org.apache.coyote.http11.Http11Protocol] (main) Initializing Coyote HTTP/1.1 on http-0.0.0.0-8080
      2010-08-02 16:21:16,403 INFO  [org.apache.coyote.ajp.AjpProtocol] (main) Initializing Coyote AJP/1.3 on ajp-0.0.0.0-8009
      2010-08-02 16:21:16,449 INFO  [org.apache.catalina.core.StandardService] (main) Starting service jboss.web
      2010-08-02 16:21:16,449 INFO  [org.apache.catalina.core.StandardEngine] (main) Starting Servlet Engine: JBoss Web/2.1.7
      2010-08-02 16:21:16,496 INFO  [org.apache.catalina.startup.Catalina] (main) Server startup in 86 ms
      2010-08-02 16:21:17,354 INFO  [org.apache.juddi.registry.RegistryServlet] (main) Loading jUDDI configuration.
      2010-08-02 16:21:17,370 INFO  [org.apache.juddi.registry.RegistryServlet] (main) Resources loaded from: /WEB-INF/juddi.properties
      2010-08-02 16:21:17,370 INFO  [org.apache.juddi.registry.RegistryServlet] (main) Initializing jUDDI components.
      2010-08-02 16:21:18,399 INFO  [org.quartz.simpl.SimpleThreadPool] (main) Job execution threads will use class loader of thread: main
      2010-08-02 16:21:18,431 INFO  [org.quartz.core.QuartzScheduler] (main) Quartz Scheduler v.1.5.2 created.
      2010-08-02 16:21:18,431 INFO  [org.quartz.simpl.RAMJobStore] (main) RAMJobStore initialized.
      2010-08-02 16:21:18,431 INFO  [org.quartz.impl.StdSchedulerFactory] (main) Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
      2010-08-02 16:21:18,431 INFO  [org.quartz.impl.StdSchedulerFactory] (main) Quartz scheduler version: 1.5.2
      2010-08-02 16:21:18,446 INFO  [org.quartz.core.QuartzScheduler] (main) Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
      2010-08-02 16:21:22,253 WARN  [org.hibernate.dialect.Oracle9Dialect] (main) The Oracle9Dialect dialect has been deprecated; use either Oracle9iDialect or Oracle10gDialect instead
      2010-08-02 16:21:25,856 INFO  [com.gehc.device.db.dao.hibernate.HibernateDAOFactory] (main) java.naming.factory.initial - org.jboss.iiop.naming.ORBInitialContextFactory
      2010-08-02 16:21:25,856 INFO  [com.gehc.device.db.dao.hibernate.HibernateDAOFactory] (main) java.naming.corba.orb - org.jacorb.orb.ORB@172aa5a
      2010-08-02 16:21:25,856 INFO  [com.gehc.device.db.dao.hibernate.HibernateDAOFactory] (main) java.naming.factory.url.pkgs - org.jboss.naming:org.jnp.interfaces:org.jboss.naming:org.jnp.interfaces
      2010-08-02 16:21:26,558 INFO  [com.gehc.device.kernel.PersistenceHelper] (main) loaded cdi configuration from [/cdi-config.xml]
      2010-08-02 16:21:26,574 INFO  [com.gehc.device.kernel.mbean.KernelService] (main) KernelFactory successfully built and bound into JNDI [java:/ecis/KernelFactory]
      2010-08-02 16:21:27,307 INFO  [com.gehc.device.util.event.SystemEventService] (main) NODE_STARTED on node [WINDOWS-XS6GOQE]
      2010-08-02 16:21:27,556 INFO  [com.gehc.device.streaminggwy.jca.AbstractConnector] (main) [aware] connector started
      2010-08-02 16:21:27,603 INFO  [com.gehc.device.streaminggwy.jca.AbstractConnector] (main) [datacaptor] connector started
      2010-08-02 16:21:27,603 INFO  [com.gehc.device.streaminggwy.jca.AbstractConnector] (main) [intellivue] connector started
      2010-08-02 16:21:27,712 INFO  [com.gehc.device.streaminggwy.jca.AbstractConnector] (main) [ixml] connector started
      2010-08-02 16:21:27,837 INFO  [com.gehc.device.util.event.SystemEventService] (main) MASTER_NODE_STARTED on node [WINDOWS-XS6GOQE]
      2010-08-02 16:21:27,853 INFO  [com.gehc.device.kernel.KernelSessionBean] (main) loaded [[null] version [null]]
      2010-08-02 16:21:28,009 INFO  [com.gehc.device.streaminggwy.jca.AbstractClientConnector] (main) the devicelink gateway retrieved from kernel is null.
      2010-08-02 16:21:33,079 INFO  [com.gehc.device.streaminggwy.jca.AbstractClientConnector] (main) the devicelink gateway retrieved from kernel is null.
      2010-08-02 16:21:33,110 INFO  [com.gehc.device.streaminggwy.jca.AbstractClientConnector] (main) the devicelink gateway retrieved from kernel is null.
      2010-08-02 16:21:33,297 INFO  [javax.enterprise.resource.webcontainer.jsf.config] (main) Initializing Mojarra (1.2_13-b01-FCS) for context '/admin-console'
      2010-08-02 16:21:36,901 INFO  [org.apache.coyote.http11.Http11Protocol] (main) Starting Coyote HTTP/1.1 on http-0.0.0.0-8080
      2010-08-02 16:21:36,932 INFO  [org.apache.coyote.ajp.AjpProtocol] (main) Starting Coyote AJP/1.3 on ajp-0.0.0.0-8009
      2010-08-02 16:23:32,384 INFO  [com.gehc.device.streaminggwy.jca.AbstractServerConnector] (http-0.0.0.0-8080-2) Adding gateway (id=[awaregateway1], name=[awaregateway1], ip=[3.204.35.9:10000] of type [CONFIG_RELOAD]
      2010-08-02 16:30:03,897 INFO  [com.gehc.device.streaminggwy.jca.AbstractServerConnector] (WorkManager(2)-3) Start processing device gateway [awaregateway1] on [3.204.35.9:10000]
      2010-08-02 21:26:30,411 ERROR [org.jgroups.protocols.UDP] (Timer-12,3.204.35.154:59962) failed sending message to null (142 bytes)
      java.lang.OutOfMemoryError: Java heap space
      at java.io.ByteArrayOutputStream.<init>(ByteArrayOutputStream.java:60)
      at org.jgroups.util.ExposedByteArrayOutputStream.<init>(ExposedByteArrayOutputStream.java:22)
      at org.jgroups.protocols.TP.send(TP.java:1460)
      at org.jgroups.protocols.TP.down(TP.java:1185)
      at org.jgroups.protocols.TP$ProtocolAdapter.down(TP.java:2304)
      at org.jgroups.protocols.Discovery.down(Discovery.java:374)
      at org.jgroups.protocols.MERGE2.down(MERGE2.java:175)
      at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:360)
      at org.jgroups.protocols.FD.down(FD.java:315)
      at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:95)
      at org.jgroups.protocols.BARRIER.down(BARRIER.java:107)
      at org.jgroups.protocols.pbcast.NAKACK.send(NAKACK.java:803)
      at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:604)
      at org.jgroups.protocols.UNICAST.down(UNICAST.java:455)
      at org.jgroups.protocols.pbcast.STABLE$1.run(STABLE.java:644)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)
      2010-08-02 21:26:52,703 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[localhost].[/config-ui-war]] (ContainerBackgroundProcessor[StandardEngine[jboss.web]]) Session event listener threw exception
      java.lang.NoClassDefFoundError: org/jboss/security/plugins/JaasSecurityManager$1
      at org.jboss.security.plugins.JaasSecurityManager.getActiveSubject(JaasSecurityManager.java:125)
      at org.jboss.web.tomcat.security.SecurityFlushSessionListener.getSubjectAndSecurityDomain(SecurityFlushSessionListener.java:184)
      at org.jboss.web.tomcat.security.SecurityFlushSessionListener.sessionDestroyed(SecurityFlushSessionListener.java:81)
      at org.apache.catalina.session.StandardSession.expire(StandardSession.java:702)
      at org.apache.catalina.session.StandardSession.isValid(StandardSession.java:592)
      at org.apache.catalina.session.ManagerBase.processExpires(ManagerBase.java:682)
      at org.apache.catalina.session.ManagerBase.backgroundProcess(ManagerBase.java:667)
      at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1327)
      at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1612)
      at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1621)
      at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1621)
      at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1601)
      at java.lang.Thread.run(Thread.java:619)
      Caused by: java.lang.ClassNotFoundException: Unexpected error during load of: org.jboss.security.plugins.JaasSecurityManager$1, msg=Java heap space
      at org.jboss.classloader.spi.base.ClassLoaderManager.process(ClassLoaderManager.java:181)
      at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:265)
      at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:1119)
      at org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:798)
      at org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:441)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
      ... 13 more
      Caused by: java.lang.OutOfMemoryError: Java heap space
      at java.nio.ByteBuffer.wrap(ByteBuffer.java:350)
      at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:137)
      at java.lang.StringCoding.decode(StringCoding.java:173)
      at java.lang.String.<init>(String.java:443)
      at sun.security.util.ManifestDigester.<init>(ManifestDigester.java:117)
      at java.util.jar.JarVerifier.processEntry(JarVerifier.java:261)
      at java.util.jar.JarVerifier.update(JarVerifier.java:199)
      at java.util.jar.JarFile.initializeVerifier(JarFile.java:323)
      at java.util.jar.JarFile.getInputStream(JarFile.java:388)
      at org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFileWrapper.java:215)
      at org.jboss.virtual.plugins.context.zip.ZipEntryContext.openStream(ZipEntryContext.java:1084)
      at org.jboss.virtual.plugins.context.zip.ZipEntryHandler.openStream(ZipEntryHandler.java:154)
      at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:241)
      at org.jboss.classloading.spi.vfs.policy.VFSClassLoaderPolicy.getResourceAsStream(VFSClassLoaderPolicy.java:483)
      at org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:534)
      at org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:532)
      at java.security.AccessController.doPrivileged(Native Method)
      at org.jboss.classloader.spi.base.BaseClassLoader.loadClassLocally(BaseClassLoader.java:530)
      at org.jboss.classloader.spi.base.BaseClassLoader.loadClassLocally(BaseClassLoader.java:507)
      at org.jboss.classloader.spi.base.BaseDelegateLoader.loadClass(BaseDelegateLoader.java:134)
      at org.jboss.classloader.spi.filter.FilteredDelegateLoader.loadClass(FilteredDelegateLoader.java:131)
      at org.jboss.classloader.spi.base.ClassLoadingTask$ThreadTask.run(ClassLoadingTask.java:452)
      at org.jboss.classloader.spi.base.ClassLoaderManager.nextTask(ClassLoaderManager.java:267)
      at org.jboss.classloader.spi.base.ClassLoaderManager.process(ClassLoaderManager.java:166)
      at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:265)
      at org.jboss.classloader.spi.base.BaseClassLoaderDomain.loadClass(BaseClassLoaderDomain.java:1119)
      at org.jboss.classloader.spi.base.BaseClassLoader.loadClassFromDomain(BaseClassLoader.java:798)
      at org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:441)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
      at org.jboss.security.plugins.JaasSecurityManager.getActiveSubject(JaasSecurityManager.java:125)
      at org.jboss.web.tomcat.security.SecurityFlushSessionListener.getSubjectAndSecurityDomain(SecurityFlushSessionListener.java:184)
      at org.jboss.web.tomcat.security.SecurityFlushSessionListener.sessionDestroyed(SecurityFlushSessionListener.java:81)
      2010-08-02 21:27:17,397 ERROR [org.jboss.ejb.plugins.LogInterceptor] (WorkManager(2)-4) TransactionRolledbackLocalException in method: public abstract void com.gehc.device.kernel.session.KernelLocalSession.sendData(com.gehc.device.model.CDIMessage) throws com.gehc.device.kernel.api.KernelException, causedBy:
      javax.ejb.EJBException: Unexpected Error

       

       

      Thanks

      Ravi S