4 Replies Latest reply on Feb 14, 2012 9:35 AM by Christophe Domas

    How to activate write-behind in file local cache store

    Christophe Domas Newbie

      Hi,

       

      I'm trying to use infinispan 5.1.1 as a persistent cache to store my application state (like a big HashMap with the good features brings by a infinspan: eviction and write-behind, listeners, etc)

      It works well but write (put) performance strongly decreased when I activate a file cache store.

       

      So, I write a little test to try different options but I think that I've missed something in write-behind concept (I read the jboss articles -  this and this ).

      In my test, I write 10 000 entries in the cache:

       

          for (int i = 0; i < COUNT; i++) {

            String id = Integer.toString(i);

            UUID randomUUID = UUID.randomUUID();

            cache.put(id, randomUUID);

          }

       

      with default config (cacheManager = new DefaultCacheManager()) it takes 400ms on my computer to put 10 000 entries (with a HashMap, it tooks 150ms the overhead is largely acceptable for me).

      But when I used the following configuration:

       

            Configuration config =

                new ConfigurationBuilder().loaders().shared(false).passivation(false).preload(false).addFileCacheStore()

                    .location(CACHE_DIR).async().threadPoolSize(10).eviction().maxEntries(100000).build();

            cacheManager = new DefaultCacheManager(config);

       

      It takes 70 seconds (it decreases to 10 seconds if I reduce the number of keys to 10 with still 10 000 values/put)

      I have set a big maxEntries to be sure that there is no conflict between write-behind and eviction.

       

      When I follow the application with a profiler, it seems the cache writing is done by the main thread, so I think the async() is misplaced but I don't find any combinaison of options to achieve my goal.

      Any idea?

       

      Best regards

       

      Christophe

        • 1. Re: How to activate write-behind in file local cache store
          Christophe Domas Newbie

          I've found how to enable... I've forgot the enable() after async() (it will be good to have toString() methods implemented on Configuration objects):

           

          Configuration config = new ConfigurationBuilder().loaders().shared(false).passivation(false).preload(false).addFileCacheStore()               .location(CACHE_DIR).async().enable().threadPoolSize(10).eviction().maxEntries(100000).build();

           

          I see a separate thread (AsyncProcessor) that write entries in the FileStore.

           

          But the test is worst (80 seconds instead 70) cause main thread spend a lot of time (30% of CPU time) in FileCacheStore.flush().

           

          Here a call tree extract from main thread:

          org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor),"2796" org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand),"2796"

          org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(InvocationContext, Object, boolean, FlagAffectedCommand),"2734"

          org.infinispan.loaders.decorators.AbstractDelegatingStore.load(Object),"2734"

          org.infinispan.loaders.LockSupportCacheStore.load(Object),"2734"

          org.infinispan.loaders.bucket.BucketBasedCacheStore.loadLockSafe(Object, Object),"2687"

          org.infinispan.loaders.bucket.BucketBasedCacheStore.loadLockSafe(Object, Integer),"2687"

          org.infinispan.loaders.file.FileCacheStore.loadBucket(Integer),"2687"

          org.infinispan.loaders.file.FileCacheStore.loadBucket(File),"2640"

          org.infinispan.loaders.file.FileCacheStore$BufferedFileSync.flush(File),"2343"

          sun.nio.ch.FileChannelImpl.force(boolean),"2343"



          Any idea?

          • 2. Re: How to activate write-behind in file local cache store
            Christophe Domas Newbie

            I finally found the performance problem. The put() operation loads the entry from cache to return the old value. This force a read from file and the flush that coast so much in my test.

            In my case, I don't need that old value.

             

            I've found that method:

                org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(InvocationContext, Object, boolean, FlagAffectedCommand)

            method use a flag to disable this read: SKIP_CACHE_STORE

             

            I was not able to find how to configure the cache to enable that but this page explains how to disable the read on a put() operation:

                cache.getAdvancedCache().withFlags(Flag.SKIP_CACHE_LOAD).put(id, randomUUID);

             

            with this option, my 10 000 entries are put on the cache in 700ms instead of 70-80 seconds!

            • 3. Re: How to activate write-behind in file local cache store
              Sanne Grinovero Master

              Hi Christophe,

              yes that's correct: sorry for the late response, but glad you found the solution.

               

              There are more Flags which provide similar options, make sure you know about all of them; I've added a link from the page you had already found.

              • 4. Re: How to activate write-behind in file local cache store
                Christophe Domas Newbie

                Thank, I was filling an enhancement ticket to have SKIP_LOAD_CACHE with configuration but you wrote a sample of how to do this with decorated cache.

                 

                I added a comment in the following page : https://docs.jboss.org/author/display/ISPN/Configuring+cache+programmatically because enable() is missing after async() in the sample.

                 

                I also filled a ticket (https://issues.jboss.org/browse/ISPN-1861) to have a toString() method in configuration objects. It should have help me to discover that write-behind was not enabled.