6 Replies Latest reply on Mar 15, 2006 12:24 AM by clebert.suconic

    Special Collections within JBossSerialization

    clebert.suconic
    clebert.suconic Mar 11, 2006 11:54 PM

      I have a need to some special collections, encapsulating ConcurrentHashMap the same way OptimizedObjectINputStream (from JBAS) uses:



      
public Object get()
{
 if (classCache != null)
 {
 WeakReference ref = (WeakReference) classCache.get(className);
 if (ref != null) clazz = (Class) ref.get();
 if (clazz == null)
 {
 if (ref != null) classCache.remove(className);

 ....
}


      Besides that one, for caching ClassLoaders I will need to create a PartitionedWeakHashMap (using WeakHashMaps with partitions, the same way ConcurrentReader does). 


      
class PartitionedHashMap extends Map
{
 WeakHashMap partitions[];
}


      I don't intend to implement the whole Map, so I don't know if that PartitionedHashMap would be useful for other projects, but I was wondering if there is any other place I could add this class, and having it available across projects. (some sort of commons/collections project).

      Also, before reinventing the wheel, anyone knows any collection library which has these kind of features?

      • 15263 Views
      • Tags:
        • 1. ClassCache (was: Special Collections within JBossSerializati
          clebert.suconic
          clebert.suconic Mar 13, 2006 11:24 PM (in response to clebert.suconic)

          Ok, as anyone else said anything, I created org.jboss.serial.util.PartitionedWeakHashMap

          I had some issues keeping refeferences of Fields.
          Every time you get a field or method from reflection, the Field is a new copy, and it holds a reference to the class.

          - If you keep a HardReference, the class is not unloaded
          - If you keep a WeakReference, the field is released too soon
          - If you keep a SoftReference, that might be the solution, but when the JVM needs memory you might loose the object.

          The solution I found was to create what I called PersistentReferences. I will be using SoftReferences, and when getting the object, if the reference is Null I am rebuilding the reference from reflection.

          with that solution I could consistently perform redeployments (tested by MemoryLeakTestCase) without leaking classes, and could recreate Fields and Method references when released by GC.

            • Actions
          • 2. Re: Special Collections within JBossSerialization
            clebert.suconic
            clebert.suconic Mar 13, 2006 11:26 PM (in response to clebert.suconic)

            Also, could use ConcurrentHashMap with WeakReferences, without leaking classLoaders

              • Actions
            • 3. Re: Special Collections within JBossSerialization
              starksm64
              starksm64 Mar 13, 2006 11:52 PM (in response to clebert.suconic)

              With a WeakReference you could not even complete a serialization op without losing the reference? What is the differenece between the ConcurrentHashMap/WeakReferences and PartitionedWeakHashMap?

                • Actions
              • 4. Re: Special Collections within JBossSerialization
                clebert.suconic
                clebert.suconic Mar 14, 2006 10:19 AM (in response to clebert.suconic)

                 

                "Scott" wrote:
                With a WeakReference you could not even complete a serialization op without losing the reference?


                With WeakReferences I would loose references on any minor garbage collection.

                PartitionedWeakHashMap, is just an implementation that is using partitions (using key.hashCode()%NUMBER_OF_PARTITIONS) to decide on what partition to use. 

                public class PartitionedWeakHashMap extends AbstractMap {

 Map[] partitionMaps;

 public PartitionedWeakHashMap()
 {
 super();
 partitionMaps = new Map[PARTITION_SIZE];
 for (int i=0;i<PARTITION_SIZE;i++)
 {
 partitionMaps = Collections.synchronizedMap(new WeakHashMap());
 }
 }


 public Map getMap(Object obj)
 {
 int hash = obj.hashCode();
 if (hash<0) hash*=-1;
 hash = hash%PARTITION_SIZE;
 return partitionMaps[hash];
 }




                  • Actions
                • 5. Re: Special Collections within JBossSerialization
                  clebert.suconic
                  clebert.suconic Mar 14, 2006 10:24 AM (in response to clebert.suconic)

                  Just an explanation on the data structure I have for WeakReferences/SoftReferences:

                  on ClassMetaModelFactory, you have the cache, as a PartitionedWeakHashMap:

                   /** PartitionedWeakHashMap<ClassLoader,ConcurrentHashMap<String,ClassMetaData>> **/
 static PartitionedWeakHashMap cache = new PartitionedWeakHashMap();


                  On ClassMetaData, I will need references to Class, Method , and ClassMetaDataField, and Constructor.

                  If you look at ClassMetaData, and ClassMetaDataField, you will have references to these reflection entities. But instead of holding a strong reference, what would avoid me redeployment, I'm holding a SoftReference, and doing some extra checks in case the reference is cleared, constructing the reference if needed.


                  This happens, because when you do Class.getMethod, Class.getField, you are actually getting a copy of Class




                    • Actions
                  • 6. Re: Special Collections within JBossSerialization
                    clebert.suconic
                    clebert.suconic Mar 15, 2006 12:24 AM (in response to clebert.suconic)

                    Just to keep a record...

                    I'm not using the PartitionWeakHashMap any more. That was just a test I was doing.

                    As WeakHashMaps are used for the ClassLoader only, it was not needed to add synchronization on that WeakHashMap

                      • Actions