8 Replies Latest reply on Mar 29, 2006 7:23 PM by ben.wang

    Comparing different serialization schemes with JBossCache

    brian.stansberry

      Discussions related to JBCACHE-515

        • 1. Re: Comparing different serialization schemes with JBossCach
          brian.stansberry

          Initial results (see JBCACHE-515) showed JGroups marshalling being more performant despite heavy synchronization.

          At least partly responsible for this is that TreeCacheMarshaller is currently written for the case where different classloaders are used for different regions of the cache. If there are no such regions, there is a fair amount of wasted overhead. See extractFqn() on the marshalling side. RegionManager.getRegion() on the demarshalling side should be pretty quick if there are no regions, but maybe I missed something.

          I wouldn't think the above would be enough to more than offset the synchronization issue, but I could be wrong.

          • 2. Re: Comparing different serialization schemes with JBossCach
            brian.stansberry

            Another thought is I wonder if there is some synchronization somewhere in the interceptor chain that is imposing enough ordering that there isn't that much contention for the JGroups output stream.

            • 3. Re: Comparing different serialization schemes with JBossCach
              belaban

              The goal here is to *not* use Util.object{To,From}ByteBuffer() anymore. We should have our own Marshaller, which knows how to marshal/unmarshall efficiently. Util.objectToByteBuffer() is a kludge, and is not even used within JGroups (we use Streamable).

              • 4. Re: Comparing different serialization schemes with JBossCach
                brian.stansberry

                +1. The inefficiencies I mention in TreeCacheMarshaller are just there because the class wasn't designed to do marshalling in all use cases; just those where a different classloader was needed for unmarshalling. They could easily be avoided if we adapted the class for general use.

                • 5. Re: Comparing different serialization schemes with JBossCach

                  Yeah, I also have reservation about the number that I obtained. It doesn't all fit yet. So I need to do more thorourgh testing to verify it.

                  But Bela is right. Even if it proves that re-use of ByteArrayOutputStream is needed for performance boost, we will also do it in TreeCacheMarshaller layer with options to deal with Region based classloader.

                  • 6. Re: Comparing different serialization schemes with JBossCach

                    So I have written another test program just to do serialization/deserialization using:

                    1. Jgroups Util.objectToByteBuffer & objectFromByteBuffer

                    2. Regular ObjectOutputStream & ObjectInputStream

                    3. JBossObjectOutputStream

                    I have run it with 20 threads no sleep with obejct size of 7.3KB. Here are the numbers:

                    1. 201 req/sec
                    2. 78 req/sec
                    3. 172 req/sec

                    Short of running thru the profiler to see why, I have done stack dump to see what happened. Here are the regular OOS dump. It turns out that most of threads are blocking at a SoftCache reference in the ObjectStreamClass class.

                    "Thread-2" prio=5 tid=0x030415d0 nid=0x98c waiting for monitor entry [0x034ef000..0x034efc68]
                    at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:243)
                    - waiting to lock <0x22f96178> (a sun.misc.SoftCache)
                    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1029)
                    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1369)
                    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1341)
                    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
                    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1073)
                    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:291)
                    at java.util.HashSet.writeObject(HashSet.java:254)

                    Meanwhile (1)'s dump showing it is blocking at:
                    "Thread-5" prio=5 tid=0x0305a008 nid=0xf48 waiting for monitor entry [0x035af000..0x035afae8]
                    at org.jgroups.util.Util.objectFromByteBuffer(Util.java:85)
                    - waiting to lock <0x22feb008> (a java.lang.Object)
                    at org.jboss.cache.TestSerial$Loader.doWorkJG(TestSerial.java:455)
                    at org.jboss.cache.TestSerial$Loader.run(TestSerial.java:336)

                    as expected.

                    Finally, if I modified (2) to use static ByteArray (as does the Util class) but no block in the ObjectInputStream part, I have got 278 req/sec.

                    • 7. Re: Comparing different serialization schemes with JBossCach
                      clebert.suconic

                      The softCache synchronization was exactly why I had started JBossSerialization.

                      I'm glad you could see the same bottleneck.

                      • 8. Re: Comparing different serialization schemes with JBossCach

                        Good. Any chance that you can take a look and optimize the JBoss Serialization further so we can beat the simple out synchronization that jgroups.util.Util employed? :-)