4 Replies Latest reply on Dec 18, 2009 4:01 AM by Galder Zamarreño

    Memory growth even after pojo is deattached from pojo cache

    Anand Somani Newbie

      Moving this from General Cache forum:

      Problem: PojoCache - huge memory usage even after detaching objects. Results in outofmemory (both Heap and non-Heap). Since it keeps increasing, bumping up the memory and permgen is not a practical solution.

      Summary: Running a stress test consisting of 3 objects. The lifespan of these objects is small (for the test) before they are removed from cache.

      Scenario:
      3 objects which need to stored - Order, Item, Attributes
      Order has => map of Items, some long fields and an enum field.
      Item => map of attributes, some long fields and an enum field.


      The flow

      1. I create an Order, add Item to it. Then attach Order to pojocache. (Not using Attributes at this point). I use the attach(String, Object) method.
      2. The order goes thru various state changes and so does Item. After order is "Completed" state it is detached from PojoCache. All done within a transaction.
      3. Since I am stress testing, I create about 4000 orders in 200 seconds, each Order takes about 10-20 seconds before it is completed and detached from PojoCache.
      4. Also note that all the changes to the objects are done using a task in a ScheduledThreadPoolExecutor. One task per Order, the task is resubmitted for each state change. I do not hold the task anywhere, after Order is "Completed" it is dereferenced.


      Observation:

      1. Heap memory contiues to grow at a steady rate peaking at abt 500 M (when the test is completed)
      2. Non Heap also continues to grow - peaking at abt 112M
      3. Almost a consistent slope for increase in memory
      4. If I run the same test without jboss cache and using an internal hashmap the memory consumption maxes at about 15 -20Mb.
      5. If I increase my load beyond a certain number I start running out of PermGen space.


      Question:

      1. I expect memory should not continue to increase, or is the test small that the cache does not get a chance to cleanup, I did try manual GC at the end of the test, but that did not make any difference in the memory footprint
      2. So am I not doing something here? I am doing a detach of obects, anything else I need to do?
      3. Also are there any know issues around Out of PermGen memory. I could not find anything on the forums.

      Test setup

      1. Java 1.6
      2. Jboss Pojo - 3.0
      3. JBoss core cache - 3.2.1
      4. Stand alone SE test.
      5. DummyTransactionManager


      I ran a small test in Yourkit and notice the top object that retains memory are ( 94MB out od 110Mb)



      Objects with biggest retained size

      Name Retained Size
      org.jboss.aop.standalone.StandaloneClassPool 55,264,688 59 %
      java.util.Hashtable 54,637,584 58 %
      java.util.Hashtable$Entry[3071] 54,637,544 58 %
      sun.misc.Launcher$AppClassLoader [Held by JVM] 13,694,848 15 %
      java.util.Vector 13,079,760 14 %
      java.lang.Object[5120] 13,079,736 14 %
      org.jboss.aop.proxy.ClassProxyFactory 10,640,456 11 %
      java.util.WeakHashMap 10,634,752 11 %
      java.util.WeakHashMap$Entry[4096] 10,634,672 11 %
      org.jboss.aop.instrument.Instrumentor 9,196,728 10 %





      Things that I plan on trying in the meantime

      * use strings instead of enum
      * compiled aspects instead of runtime weaving
      * Use 3.0.3 version of pojocache instead of 3.2.1




      Thanks
      Anand