How to activate write-behind in file local cache store

[domak]

Hi,

I'm trying to use infinispan 5.1.1 as a persistent cache to store my application state (like a big HashMap with the good features brings by a infinspan: eviction and write-behind, listeners, etc)

It works well but write (put) performance strongly decreased when I activate a file cache store.

So, I write a little test to try different options but I think that I've missed something in write-behind concept (I read the jboss articles -  this and this ).

In my test, I write 10 000 entries in the cache:

    for (int i = 0; i < COUNT; i++) {

      String id = Integer.toString(i);

      UUID randomUUID = UUID.randomUUID();

      cache.put(id, randomUUID);

    }

with default config (cacheManager = new DefaultCacheManager()) it takes 400ms on my computer to put 10 000 entries (with a HashMap, it tooks 150ms the overhead is largely acceptable for me).

But when I used the following configuration:

      Configuration config =

          new ConfigurationBuilder().loaders().shared(false).passivation(false).preload(false).addFileCacheStore()

              .location(CACHE_DIR).async().threadPoolSize(10).eviction().maxEntries(100000).build();

      cacheManager = new DefaultCacheManager(config);

It takes 70 seconds (it decreases to 10 seconds if I reduce the number of keys to 10 with still 10 000 values/put)

I have set a big maxEntries to be sure that there is no conflict between write-behind and eviction.

When I follow the application with a profiler, it seems the cache writing is done by the main thread, so I think the async() is misplaced but I don't find any combinaison of options to achieve my goal.

Any idea?

Best regards

Christophe

[domak]

I've found how to enable... I've forgot the enable() after async() (it will be good to have toString() methods implemented on Configuration objects):

Configuration config = new ConfigurationBuilder().loaders().shared(false).passivation(false).preload(false).addFileCacheStore()               .location(CACHE_DIR).async().enable().threadPoolSize(10).eviction().maxEntries(100000).build();

I see a separate thread (AsyncProcessor) that write entries in the FileStore.

But the test is worst (80 seconds instead 70) cause main thread spend a lot of time (30% of CPU time) in FileCacheStore.flush().

Here a call tree extract from main thread:

org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor),"2796" org.infinispan.interceptors.CacheLoaderInterceptor.visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand),"2796"

org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(InvocationContext, Object, boolean, FlagAffectedCommand),"2734"

org.infinispan.loaders.decorators.AbstractDelegatingStore.load(Object),"2734"

org.infinispan.loaders.LockSupportCacheStore.load(Object),"2734"

org.infinispan.loaders.bucket.BucketBasedCacheStore.loadLockSafe(Object, Object),"2687"

org.infinispan.loaders.bucket.BucketBasedCacheStore.loadLockSafe(Object, Integer),"2687"

org.infinispan.loaders.file.FileCacheStore.loadBucket(Integer),"2687"

org.infinispan.loaders.file.FileCacheStore.loadBucket(File),"2640"

org.infinispan.loaders.file.FileCacheStore$BufferedFileSync.flush(File),"2343"

sun.nio.ch.FileChannelImpl.force(boolean),"2343"

Any idea?

[domak]

I finally found the performance problem. The put() operation loads the entry from cache to return the old value. This force a read from file and the flush that coast so much in my test.

In my case, I don't need that old value.

I've found that method:

    org.infinispan.interceptors.CacheLoaderInterceptor.loadIfNeeded(InvocationContext, Object, boolean, FlagAffectedCommand)

method use a flag to disable this read: SKIP_CACHE_STORE

I was not able to find how to configure the cache to enable that but this page explains how to disable the read on a put() operation:

    cache.getAdvancedCache().withFlags(Flag.SKIP_CACHE_LOAD).put(id, randomUUID);

with this option, my 10 000 entries are put on the cache in 700ms instead of 70-80 seconds!

[sannegrinovero]

Hi Christophe,

yes that's correct: sorry for the late response, but glad you found the solution.

There are more Flags which provide similar options, make sure you know about all of them; I've added a link from the page you had already found.

[domak]

Thank, I was filling an enhancement ticket to have SKIP_LOAD_CACHE with configuration but you wrote a sample of how to do this with decorated cache.

I added a comment in the following page : https://docs.jboss.org/author/display/ISPN/Configuring+cache+programmatically because enable() is missing after async() in the sample.

I also filled a ticket (https://issues.jboss.org/browse/ISPN-1861) to have a toString() method in configuration objects. It should have help me to discover that write-behind was not enabled.