6 Replies Latest reply on Mar 15, 2006 12:24 AM by clebert.suconic

    Special Collections within JBossSerialization

    clebert.suconic

      I have a need to some special collections, encapsulating ConcurrentHashMap the same way OptimizedObjectINputStream (from JBAS) uses:



      
      public Object get()
      {
       if (classCache != null)
       {
       WeakReference ref = (WeakReference) classCache.get(className);
       if (ref != null) clazz = (Class) ref.get();
       if (clazz == null)
       {
       if (ref != null) classCache.remove(className);
      
       ....
      }
      
      


      Besides that one, for caching ClassLoaders I will need to create a PartitionedWeakHashMap (using WeakHashMaps with partitions, the same way ConcurrentReader does).


      
      class PartitionedHashMap extends Map
      {
       WeakHashMap partitions[];
      }
      
      


      I don't intend to implement the whole Map, so I don't know if that PartitionedHashMap would be useful for other projects, but I was wondering if there is any other place I could add this class, and having it available across projects. (some sort of commons/collections project).

      Also, before reinventing the wheel, anyone knows any collection library which has these kind of features?

        • 1. ClassCache (was: Special Collections within JBossSerializati
          clebert.suconic

          Ok, as anyone else said anything, I created org.jboss.serial.util.PartitionedWeakHashMap

          I had some issues keeping refeferences of Fields.
          Every time you get a field or method from reflection, the Field is a new copy, and it holds a reference to the class.

          - If you keep a HardReference, the class is not unloaded
          - If you keep a WeakReference, the field is released too soon
          - If you keep a SoftReference, that might be the solution, but when the JVM needs memory you might loose the object.

          The solution I found was to create what I called PersistentReferences. I will be using SoftReferences, and when getting the object, if the reference is Null I am rebuilding the reference from reflection.

          with that solution I could consistently perform redeployments (tested by MemoryLeakTestCase) without leaking classes, and could recreate Fields and Method references when released by GC.

          • 2. Re: Special Collections within JBossSerialization
            clebert.suconic

            Also, could use ConcurrentHashMap with WeakReferences, without leaking classLoaders

            • 3. Re: Special Collections within JBossSerialization
              starksm64

              With a WeakReference you could not even complete a serialization op without losing the reference? What is the differenece between the ConcurrentHashMap/WeakReferences and PartitionedWeakHashMap?

              • 4. Re: Special Collections within JBossSerialization
                clebert.suconic

                 

                "Scott" wrote:
                With a WeakReference you could not even complete a serialization op without losing the reference?


                With WeakReferences I would loose references on any minor garbage collection.

                PartitionedWeakHashMap, is just an implementation that is using partitions (using key.hashCode()%NUMBER_OF_PARTITIONS) to decide on what partition to use.

                public class PartitionedWeakHashMap extends AbstractMap {
                
                 Map[] partitionMaps;
                
                 public PartitionedWeakHashMap()
                 {
                 super();
                 partitionMaps = new Map[PARTITION_SIZE];
                 for (int i=0;i<PARTITION_SIZE;i++)
                 {
                 partitionMaps = Collections.synchronizedMap(new WeakHashMap());
                 }
                 }
                
                
                 public Map getMap(Object obj)
                 {
                 int hash = obj.hashCode();
                 if (hash<0) hash*=-1;
                 hash = hash%PARTITION_SIZE;
                 return partitionMaps[hash];
                 }
                




                • 5. Re: Special Collections within JBossSerialization
                  clebert.suconic

                  Just an explanation on the data structure I have for WeakReferences/SoftReferences:

                  on ClassMetaModelFactory, you have the cache, as a PartitionedWeakHashMap:

                   /** PartitionedWeakHashMap<ClassLoader,ConcurrentHashMap<String,ClassMetaData>> **/
                   static PartitionedWeakHashMap cache = new PartitionedWeakHashMap();
                  


                  On ClassMetaData, I will need references to Class, Method , and ClassMetaDataField, and Constructor.

                  If you look at ClassMetaData, and ClassMetaDataField, you will have references to these reflection entities. But instead of holding a strong reference, what would avoid me redeployment, I'm holding a SoftReference, and doing some extra checks in case the reference is cleared, constructing the reference if needed.


                  This happens, because when you do Class.getMethod, Class.getField, you are actually getting a copy of Class




                  • 6. Re: Special Collections within JBossSerialization
                    clebert.suconic

                    Just to keep a record...

                    I'm not using the PartitionWeakHashMap any more. That was just a test I was doing.

                    As WeakHashMaps are used for the ClassLoader only, it was not needed to add synchronization on that WeakHashMap