when does java release commited memory back to the OS

[hari.svnit]

We have a Jboss server running on a 64-bit windows server with jdk6(64 bit) .
Here is a snapshot the memory usage as viewed from jconsole.

Heap Memory Usage :
Used : 103 MB, Committed : 295 MB, Max : 571 MB

Non-Heap Memory Usage:
Used : 47MB, Committed : 47 MB, Max : 115 MB

Memory Usage as viewed from task manager : 314 MB
Running GC did not have much of an effect on the numbers.

The total used memory is 150 MB but the total committed memory is 342 MB. How does JVM decide on the amount of committed memory ?

Here are the jvm options:
-verbosegc -Xloggc:gc.log -Xms150m -Xmx600m -Xmn125m -XX:+UseParallelGC -XX:ParallelGCThreads=20 -XX:MaxHeapFreeRatio=40 -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Dsun.rmi.dgc.client.gcInterval=60000 -Dsun.rmi.dgc.server.gcInterval=60000

We are planning to deploy more such jboss server configurations on the same machine. So we are afraid we may run out of memory. Can somebody please help us out ?

[peterj]

According to your heap settings, the JVM will use at least 150MB for the heap, but can use as much as 600MB. Apparently, the JVM has at this time increased the heap size to 295MB, of which only 103MB is in use.

The non-heap memory usage is for the thread stacks, the JVM's C/C++ data structures and other odds and ends.

By the way, your new size setting (-Xmn) practically guarantees full GCs every time, and full GCs are about way more expensive (10 - 100 times) than minor GCs. You should set the new size to at most 1/3 the size of the heap.

See my presentation on the Java heap: http://www.cecmg.de/doc/tagung_2007/agenda07/24-mai/2b3-peter-johnson/index.html

[hari.svnit]

Hi Peter, Thanks a lot for the super-quick reply. I have gone through your article and I must admit that it is top class.
I have tried with the new setting you suggested for -Xmn. I have tried the following test this time :
I have created two jboss server configurations, and i have a code(in index.jsp of the web-app) which takes the heap memory usage to 500MB while the jsp is rendered, and (from jconsole) this usage is coming down to 120 MB after gc.
But the committed memory is still 500 MB and in the task manager the usage is still at the peak of 550 MB.
The same is happening with the second server configuation as well.

So Although, both my java processes together are using only 300 MB at the moment, the task manager sees them as using 1100MB together and my physical memory is getting 94% full.
In this case, since the used memory is coming down to its startup value, I am guessing it is not either a memory leak or garbage collection issue.

How do i get these processes to release the extra memory ? Can you please help us out ?

Thanks
Hari

[peterj]

Once the JVM asks the OS for memory for the heap it is reluctant to let it go. If you want to limit the amount of memory used, restrict the max heap size. Your only other option is to get more RAM (or allocate larger page files).

[hari.svnit]

Peter, Does this mean that the amount of memory held/locked/committed by a jvm is always growing no matter what is actually needed/used ( at least in case of windows OS) ? ( I have no more questions if the answer is yes :) )

Thanks
Hari

[peterj]

Pretty much, yes. Under some circumstances the JVM will reduce the heap size. At least, that is what my coworker who has worked with that code in the JVM says, but practically I have never witnessed the heap size decreasing.

[noelo]

Hi Peter,
Just a query on what you have said above i.e.

By the way, your new size setting (-Xmn) practically guarantees full GCs every time, and full GCs are about way more expensive (10 - 100 times) than minor GCs. You should set the new size to at most 1/3 the size of the heap.

Given the following settings

-Xms150m -Xmx600m -Xmn125m

Why would it perform a full GC ? Is it because the Xms is so small ?
If the Xms and Xmx were the same i.e. 600M would the Xmn settings be ok ?

I don't mean to hijack the thread just looking for some help understanding the implications of the Xmn setting.

regards
Noel

[peterj]

Using the default collectors (CMS and G1 are different), GC does not take place until there is no room in the eden space (within the young generation) to allocate an object. The first thing the JVM does is determine if it needs to collect the old generation before it collects the young generation. The way the Sun JVM does this is it assumes the worst-case scenario that all objects currently occupying the young generation will survive the collection, and thus if the size of the young generation is less than the available space in the old generation, it collects the old generation first, and then the young generation (thus resulting in a full collection).

Example using your settings and assuming the JVM starts with the min heap size. You are starting with:

young gen: 125MB, old gen: 25MB

Hmm, that is not good - the first collection might result in a full collection but the old gen is empty. My bet is that the heap will immediately be resized, so let's pick a better heap size, say 300MB, then we have:

young gen: 125MB, old gen: 175MB

Let's say that for the first minor collection 60MB of objects are placed into the old gen (usually not likely, but after several collections you could have that much space used). So now you have only 115MB of free space left in the old gen. The next collection will be a full collection (125MB young gen > 115MB free old gen). Now let's assume the JVM does not adjust the heap size any more, and that there is always around 60MB of objects occupying the old gen. The result: every collection is a full collection.

Now, if the JVM resizes the heap up to the 600MB max, you are in better shape. Then you have:

young gen: 125MB, old gen: 475MB

Not until the old gen contains > 350MB of objects would the JVM do a major collection. Of course, if your working set size is > 350MB then you will find yourself in the same boat again - major collection each time.

Now in the above I have not taken the size of the survivor spaces into account, nor any fine tuning the that JVM does with heap sizing during each collection, or advances in newer JVMs (an early version of JDK5 was the last one I checked this on). You can easily monitor the exact behavior by turning on one of the GC statistics options.

[noelo]

Thanks for the info Peter,

regards
Noel

[frankgrimes97]

You might want to have a look at http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6498735

Specifically the comment which explains "-XX:[Max|Min]HeapFreeRatio"

[somostetoikilato]

Hi Peter,

Where can I find your presentation about the heap memory?

The URL http://www.cecmg.de/doc/tagung_2007/agenda07/24-mai/2b3-peter-johnson/index.html is invalid.

Could you send it to me?

Thank you.

SK

[peterj]

Here is an updated link that has my various white papers in JVM performance and tuning:

http://www.cmg.org/cgi-bin/search.cgi?q=java+peter+johnson&x=30&y=10

Look for these white papers:

* Java Garbage Collection Statistical Analysis 101

* Java Garbage Collection Performance Analysis 201