Nobody can't give you the ultimate answer how to set the Heap and GC.
It depends extremely to your application, so it is your housework.
The first thing is to check whether you have an application memory leak. This can be done by monitoring the JVM.
If you see that the OldGen utilization is growing during the time (and the throughput is not increasing as well) you might use a profiler (testenv) or heapdumps and compare it.
From what you post it looks like a leak.