3.0.0 was the first 3 release and as you expect things have quickly gotten a lot better since then. Try a later version and see if you have better luck.
I tried the same application on Version 3.0.3 with Tomcat embedded and I got the same results.
I find that after the application has run I get a exception from JBoss code:
javax.ejb.EJBException: Could not passivate; CausedByException is:
If JBoss can't passivate some of its components, maybe it can't retreive some memory ??
It looks like you are using JBoss MQ (are you ???) which has known memory leaks, and should be fixed in 3.0.4