Memory issues with JBoss 4.2.2.GA

[bngsvcs]

I have JBoss 4.2.2.GA installed on Win 2003 Server along with MySQL 5.0 and Apache 2.0. Nearly 2 weeks ago I deployed my Java application and the website went live.

We had over 500,000 hits the first day and have had as many as 750,000 hits in a day since. I have also encountered numerous memory problems. The memory problems have varied from out of heap to GC out of memory.

After a great deal of research I have the JVM parameter in run.conf set to:

JAVA_OPTS="-Xms1024M -Xmx2048M --XX:PermSize=512m -XX:MaxPermSize=512m -XX:+UseConcMarkSweepGC -XX:+CMSPermGenSweepingEnabled -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC"

I am still having memory problems.

If I bring up the JBoss web console it tells me I have 123M of free memory. I have monitored the service and it shows a minimum allocation of 64M of memory and never goes above 128M.

I did go back and modify run.bat with the same parameters as above and the results did not change.

I realize I may still need to tune the settings for proper use of the heap but I first need to see that JBoss is receiving a better allocation of space.

My owner is giving me holy he## at the moment. I need to get this monster tuned so that it stops running out of memory but I can't seem to make a difference in how much space it is being allocated.

What am I doing wrong here????

[peterj]

You are printing out the detailed GC information. Have you graphed those to see what the heap usage looks like? For hints on graphing, see my presentation at http://www.cecmg.de/doc/tagung_2007/agenda07/24-mai/2b3-peter-johnson/index.html.

Also, you are using the CMS collector, so pay attention to the slide on CMS gotchas.

[bngsvcs]

"PeterJ" wrote:
You are printing out the detailed GC information. Have you graphed those to see what the heap usage looks like? For hints on graphing, see my presentation at http://www.cecmg.de/doc/tagung_2007/agenda07/24-mai/2b3-peter-johnson/index.html.

Also, you are using the CMS collector, so pay attention to the slide on CMS gotchas.

I appreciate the response. And I will be reviewing those topics. But I still need to understand why the JBoss processes do not appear to be getting the space that I have attempted to allocate.

If I allocate 3GB to the JVM in the run.conf file why does the web-console tell me it only has 124MB? 124MB is not enough for this web application.

[peterj]

Do not rely on the data given by web-console. Use JConsole or VirtualVM to monitor real-time heap usage. Use my presentation to graph the GC data you are collecting and to help you tune the heap.

Since you set the heap to 1GB minimum, the JVM will allocate that much space. Not all will be used, In fact, the CMS collector will use a lot less of it than the default collector due to the way the CMS collector works (my presentation explains that also).

P.S. You do not have to quote the post.

[bngsvcs]

I just made an interesting discovery. Since this is running on a Windows platform I am using JBossService. I did not realize that it makes entries on the registry. It turns out that the parameters on the registry is what is being allocated to JBoss. What I found there was 64MB and 128MB. I changed them to 128MB and 256 MB and the web-console now shows the new settings.

In addition to this I had noticed that the web-console was showing a back level JVM. Settings for that back level JVM were also on the registry.

I really hadn't thought of the service causing a big difference between what I am running and what the documentation is describing. Obviously I needed to. So at this point I have to assume that the settings in the run.conf file don't mean anything when running JBossService.

I'll have to start researching how to change the configuration with JBossService.

[peterj]

Sounds like you are using the JavaService tool from http://javaservice.objectweb.org/. It has a script that creates a service named JBossService and places all configuration information in the registry. Note that that project has nothing to do with JBoss AS.

As you found out, omitting key information such as this is likely to generate no-so-helpful responses.

Finally, use of JavaService is discouraged, see http://www.jboss.org/community/docs/DOC-11932, first item under "Notes"