Do you really needs 16GB of heap? Have you done any performance testing to see what the actual memory requirements are? Cleaning up a 16GB heap, or even a 1GB young generation, will take time. And using the CMS collector will not reduce the young generation pauses. The CMS collector does pause the app twice for object marking; I have no idea how long it will take to mark a 16 GB heap.
Ideally, you want the smallest heap possible that does not greatly negatively impact your performance. In this case I think you have run into the "more is not better" anti-pattern.
reason i have heap set high was to avoid out of memory exceptions.
Whats the best way to use the 24g RAM that is available.
We are running lot of Web Apps and they all need to store large amount of data such as session objects etc thats why i am using 16g heap.
If i restrict the heap size to say 4g, than there is 20G ram not used by the server at all. Is this assumption correct?
Yes, if JBoss AS is the only thing running, and you allocate only a 4GB heap, then around 20GB of RAM will be unused. Actually, more like 18 or 19GB because the OS and various services will use some memory, the Java threads will use memory, the JVM will use memory, etc. There were some recent posts on how he JVM consumes memory.
I still stand by my statements on heap sizing - a too large heap is as bad as a too small heap. There is an ideal heap size for your apps, but without monitoring the heap usage you will never know what it is.