That mght not consumed by the java process, but Linux might use free memory to cache things. Only issue is if SWAP is used, then the physical memory is swapped to disc which is mostly a performance killer.
You should check waht the java process consume.
Also I recommend to use a Java7 or 8 JVM to have the latest fixes.
Thanks Fink for your reply. I checked java process from top command its showing RES: 5gb and less. And i stopped jboss for 2 days still linux RAM is not reduced so i just rebooted linux machine and started jboss again after some time RAM reached to 40+ GB. How do i prove/check that SWAP is consuming that memory ?. Moreover in TOP command SWAP consumed memory is zero.I will go with java 7 version as you suggested. Thanks once again Fink.