Do you monitor the GC with GC logging? Also you might use "jstat -gccause <pid>" to have a view how the memory is used.
Could it be application related? Maybe more load or batch at this time?
Sometimes the reason is a bit before the server starts getting slow.
I don't see whether the server itself can be the reason, but you might upgrade to a newer version as this version is older and there are bugfixes and enhancements which might help.
There are some issues with EJB2, but I'm not aware of such one. All these issues are more functional and does not end in such decrase after a time.
As there is a lot of time between AS3.2 and AS7 there are lots of changes which might have bugs or cause issues which need updates.
You need to elaborate what the reason is by checking whether it is a memory issue or something else.