It would appear that the class loaders are hanging on to several classes/instances, which in turn are holding on to the consumed memory. After all, the GC will not free up objects that are still in use (i.e., being referenced by other objects).
What you need to do is take a heap dump and see what is using up all of the space. You can use VisualVM to do this. It comes with JDK 6 as jvisualvm (and yes you can use it to diagnose an app runnign with JDK 5), or you can download it from https://visualvm.dev.java.net/download.html. Or you can set the -XX:+HeapDumpOnOutOfMemoryError JVM option, which will do as it says and then load the dump into VisualVM.
Then there is also jhat and jmap, but that will work only on Linux for JDK 5.
Also, when loading the heap dump, give yourself about 3x the amount of heap as what you dumped. For example, you have a 2GB heap, you will need to allocate a 6GB heap for VisualVM to analyze that heap dump, which means you will need a 64-bit JVM.