6 Replies Latest reply on Sep 4, 2007 9:10 AM by nicdo77

    Java heap size problem

    nicdo77

      Hi,

      I'm using JBoss 4.0.5 and i have two applications deployed on it: the application i'm developing and JIRA.
      Since i have installed JIRA, i have had performance problems, such as gettin this in the log:

      15:20:37,984 ERROR [STDERR] Exception in thread "ScannerThread"
       15:20:38,703 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
       Server daemon died!
      15:21:00,937 ERROR [STDERR] at java.util.TreeMap.buildFromSorted(TreeMap.java:1611)
      15:21:02,781 ERROR [STDERR] at java.util.TreeMap.buildFromSorted(TreeMap.java:1548)
      15:21:02,781 ERROR [STDERR] at java.util.TreeMap.putAll(TreeMap.java:320)
      15:21:02,781 ERROR [STDERR] at javax.management.modelmbean.DescriptorSupport.init(DescriptorSupport.java:467)
      15:21:02,781 ERROR [STDERR] at javax.management.modelmbean.DescriptorSupport.<init>(DescriptorSupport.java:210)
      15:21:02,781 ERROR [STDERR] at javax.management.modelmbean.DescriptorSupport.clone(DescriptorSupport.java:801)
      15:21:02,781 ERROR [STDERR] at javax.management.modelmbean.ModelMBeanAttributeInfo.getDescriptor(ModelMBeanAttributeInfo.java:348)
      15:21:02,781 ERROR [STDERR] at javax.management.modelmbean.ModelMBeanAttributeInfo.<init>(ModelMBeanAttributeInfo.java:316)
      15:21:02,781 ERROR [STDERR] at javax.management.modelmbean.ModelMBeanAttributeInfo.clone(ModelMBeanAttributeInfo.java:407)
      15:21:02,781 ERROR [STDERR] at javax.management.modelmbean.ModelMBeanInfoSupport.getAttribute(ModelMBeanInfoSupport.java:731)
      15:21:02,781 ERROR [STDERR] at javax.management.modelmbean.ModelMBeanInfoSupport.getDescriptor(ModelMBeanInfoSupport.java:544)
      15:21:02,781 ERROR [STDERR] at org.jboss.mx.modelmbean.ModelMBeanInvoker.override(ModelMBeanInvoker.java:830)
      15:21:02,796 ERROR [STDERR] at org.jboss.mx.server.AbstractMBeanInvoker.getAttribute(AbstractMBeanInvoker.java:350)
      15:21:02,796 ERROR [STDERR] at org.jboss.mx.server.MBeanServerImpl.getAttribute(MBeanServerImpl.java:556)
      15:21:02,796 ERROR [STDERR] at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:140)
      15:21:02,796 ERROR [velocity] Method getSelectListOptions threw exception for reference $searcher in template templates/jira/issue/searchers/edit/user-searcher-edit.vm at [148,30]
      15:21:02,796 ERROR [STDERR] at $Proxy11.getSuffixOrder(Unknown Source)
      15:21:02,796 ERROR [STDERR] at org.jboss.deployment.scanner.URLDeploymentScanner.updateSorter(URLDeploymentScanner.java:671)
      15:21:02,796 ERROR [DefaultVelocityManager] MethodInvocationException occurred getting message body from Velocity: java.lang.OutOfMemoryError: Java heap space
       java.lang.OutOfMemoryError: Java heap space
      15:21:02,796 ERROR [STDERR] at org.jboss.deployment.scanner.URLDeploymentScanner.scan(URLDeploymentScanner.java:640)
      15:21:02,796 ERROR [STDERR] at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.doScan(AbstractDeploymentScanner.java:263)
      15:21:02,796 ERROR [STDERR] at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.loop(AbstractDeploymentScanner.java:274)
      15:21:02,796 ERROR [STDERR] at org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread.run(AbstractDeploymentScanner.java:225)
      15:21:02,796 ERROR [STDERR] Exception in thread "JiraQuartzScheduler_Worker-0"
      15:21:02,796 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
      15:21:02,796 ERROR [STDERR] at java.nio.ByteBuffer.wrap(ByteBuffer.java:350)
      15:21:02,796 ERROR [STDERR] at java.lang.StringCoding$CharsetSD.decode(StringCoding.java:187)
      15:21:02,796 ERROR [STDERR] at java.lang.StringCoding.decode(StringCoding.java:228)
      15:21:02,796 ERROR [STDERR] at java.lang.StringCoding.decode(StringCoding.java:234)
      15:21:02,796 ERROR [STDERR] at java.lang.String.<init>(String.java:458)
      15:21:02,796 ERROR [STDERR] at org.tanukisoftware.wrapper.WrapperManager.handleSocket(WrapperManager.java:3759)
      15:21:02,796 ERROR [STDERR] at org.tanukisoftware.wrapper.WrapperManager.run(WrapperManager.java:4084)
      15:21:02,796 ERROR [STDERR] at java.lang.Thread.run(Thread.java:595)
      15:21:02,796 ERROR [STDERR] Exception in thread "HSQLDB Timer @75be38"
      15:21:02,796 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
      15:21:02,796 ERROR [STDERR] at java.lang.String.<init>(String.java:208)
      15:21:02,796 ERROR [STDERR] at java.lang.StringBuffer.toString(StringBuffer.java:586)
      15:21:02,796 ERROR [STDERR] at org.apache.log4j.PatternLayout.format(PatternLayout.java:498)
      15:21:02,796 ERROR [STDERR] at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:292)
      15:21:03,437 ERROR [STDERR] at org.apache.log4j.DailyRollingFileAppender.subAppend(DailyRollingFileAppender.java:349)
      15:21:03,437 ERROR [STDERR] at org.apache.log4j.WriterAppender.append(WriterAppender.java:150)
      15:21:03,437 ERROR [STDERR] at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:221)
      15:21:03,437 ERROR [STDERR] at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:57)
      15:21:03,437 ERROR [STDERR] at org.apache.log4j.Category.callAppenders(Category.java:187)
      15:21:03,437 ERROR [STDERR] at org.apache.log4j.Category.forcedLog(Category.java:372)
      15:21:03,437 ERROR [STDERR] at org.apache.log4j.Category.debug(Category.java:241)
      15:21:03,437 ERROR [STDERR] at com.atlassian.jira.service.ServiceRunner.execute(ServiceRunner.java:96)
      15:21:03,437 ERROR [STDERR] at org.quartz.core.JobRunShell.run(JobRunShell.java:191)
      15:21:03,437 ERROR [STDERR] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
      15:21:03,437 ERROR [STDERR] java.lang.OutOfMemoryError: Java heap space
      15:21:07,375 ERROR [velocity] Method getSelectListOptions threw exception for reference $searcher in template templates/jira/issue/searchers/edit/user-searcher-edit.vm at [34,22]
      15:21:07,375 ERROR [DefaultVelocityManager] MethodInvocationException occurred getting message body from Velocity: java.lang.OutOfMemoryError: Java heap space
      java.lang.OutOfMemoryError: Java heap space
      15:21:07,390 INFO [velocity] ResourceManager : found templates/jira/issue/searchers/edit/constants-searcher-edit.vm with loader org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader
      15:21:10,875 ERROR [velocity] Method getText threw exception for reference $i18n in template templates/jira/issue/searchers/edit/constants-searcher-edit.vm at [3,43]
      15:21:10,875 ERROR [DefaultVelocityManager] MethodInvocationException occurred getting message body from Velocity: java.lang.OutOfMemoryError: Java heap space
      java.lang.OutOfMemoryError: Java heap space
      15:21:16,312 ERROR [ServletValueStack] METHOD: "searcherEditHtml", exception:
       java.lang.OutOfMemoryError: Java heap space
       JVM appears hung: Timed out waiting for signal from JVM.
       JVM did not exit on request, terminated
       Launching a JVM...
      Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
       Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
      


      As you can see, i'm using JBoss as a Service Windows using a wrapper (see here for more details:
      http://wrapper.tanukisoftware.org/doc/english/jmx.html#jboss)

      So i have the following wrapper.conf:

      #********************************************************************
      # Wrapper Properties
      #********************************************************************
      # Java Application
      wrapper.java.command=%JAVA_HOME%\bin\java
      
      # Java Main class. This class must implement the WrapperListener interface
      # or guarantee that the WrapperManager class is initialized. Helper
      # classes are provided to do this for you. See the Integration section
      # of the documentation for details.
      wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp
      
      # Java Classpath (include wrapper.jar) Add class path elements as
      # needed starting from 1
      wrapper.java.classpath.1=%JBOSS_HOME%/server/default/lib/wrapper.jar
      wrapper.java.classpath.2=%JAVA_HOME%/lib/tools.jar
      wrapper.java.classpath.3=./run.jar
      
      # Java Library Path (location of Wrapper.DLL or libwrapper.so)
      wrapper.java.library.path.1=%JBOSS_HOME%/server/default/lib
      
      # Java Additional Parameters
      #wrapper.java.additional.1=
      
      # Initial Java Heap Size (in MB)
       #wrapper.java.initmemory=1024
      
       # Maximum Java Heap Size (in MB)
       #wrapper.java.maxmemory=2048
      
      # Application parameters. Add parameters as needed starting from 1
      wrapper.app.parameter.1=org.jboss.Main
      
      #********************************************************************
      # Wrapper Logging Properties
      #********************************************************************
      # Format of output for the console. (See docs for formats)
      wrapper.console.format=M
      
      # Log Level for console output. (See docs for log levels)
      wrapper.console.loglevel=INFO
      
      # Log file to use for wrapper output logging.
      wrapper.logfile=%JBOSS_HOME%/server/default/log/wrapper.log
      
      # Format of output for the log file. (See docs for formats)
      wrapper.logfile.format=M
      
      # Log Level for log file output. (See docs for log levels)
      wrapper.logfile.loglevel=INFO
      
      # Maximum size that the log file will be allowed to grow to before
      # the log is rolled. Size is specified in bytes. The default value
      # of 0, disables log rolling. May abbreviate with the 'k' (kb) or
      # 'm' (mb) suffix. For example: 10m = 10 megabytes.
      wrapper.logfile.maxsize=10m
      
      # Maximum number of rolled log files which will be allowed before old
      # files are deleted. The default value of 0 implies no limit.
      wrapper.logfile.maxfiles=10
      
      # Log Level for sys/event log output. (See docs for log levels)
      wrapper.syslog.loglevel=NONE
      
      #********************************************************************
      # Wrapper Windows Properties
      #********************************************************************
      # Title to use when running as a console
      wrapper.console.title=@app.long.name@
      
      #********************************************************************
      # Wrapper Windows NT/2000/XP Service Properties
      #********************************************************************
      # WARNING - Do not modify any of these properties when an application
      # using this configuration file has been installed as a service.
      # Please uninstall the service before modifying this section. The
      # service can then be reinstalled.
      
      # Name of the service
      wrapper.ntservice.name=JBoss Opalys
      
      # Display name of the service
      wrapper.ntservice.displayname=JBoss Opalys
      
      # Description of the service
      wrapper.ntservice.description=J2EE Application Server for Opalys Application
      
      # Service dependencies. Add dependencies as needed starting from 1
      wrapper.ntservice.dependency.1=
      
      # Mode in which the service is installed. AUTO_START or DEMAND_START
      wrapper.ntservice.starttype=AUTO_START
      
      # Allow the service to interact with the desktop.
      wrapper.ntservice.interactive=false


      As you can see, i set the minimal java memory size at 1024 and the max at 2048. Thinking it should be enough :)

      But the point is, i keep getting Java heap size problems (as i said earlier) and i don't know where they are coming. I don't think my apps are resource-consuming: they work perfectly locally, i can let my JBoss turn all day long without any problems, inside or outside my IDE.
      But once it came to production, i get problems.

      So, i had a look at the JMX Console, and in the Mbean {Jboss.system, type=ServerInfo}, i get the following data:

      FreeMemory java.lang.Long R 42816 MBean Attribute.
       TotalMemory java.lang.Long R 66650112 MBean Attribute.
       MaxMemory java.lang.Long R 66650112 MBean Attribute.


      So apparently from this data i understand that the actual memory of my JBoss is only 64mb... but why???
      Please help me