13 Replies Latest reply on Jun 30, 2009 6:31 AM by galder.zamarreno

    Async Eviction and Correct Cache Usage

    amin59

      Thanks for the great product. I have just started testing infinispan and this is my first post in the list.

      Sorry if my question are vivid. (maybe this is because of docs lacking, which I hope to be fixed and enriched asap).

      1. How can I config inifispan to persis my object asynchronously and not as soon as they are change. I have developed a JDBC cache store for myself (which works well) but evict or persist is still sync. here is my config:

       config.setEvictionStrategy(EvictionStrategy.FIFO);
       config.setEvictionWakeUpInterval(60 * 1000);
       config.setCacheMode(Configuration.CacheMode.INVALIDATION_ASYNC);
      


      2. I have a counter/billing like object in which in every system event some counters are increased and some values (e.g. credit) is decreased. This happens hundred/thousand times in a second and should be coherent across all nodes. What is the correct policy for this case. Should I rebuild a new object every time and replace it in the cache with the previous one? This solution will require me to create an object many times. Is there any other (better) solution?

      3. May I have a complete sample of using transactions (including boilerplate code) how to get transaction object. I need this for question #2 to work correctly.

      4. AFAIK distributed locks are not implemented yet. When will they be available?

        • 1. Re: Async Eviction and Correct Cache Usage
          manik

          1. Is your cache store configured to be async? See CacheStoreConfig.setAsyncCacheStoreConfig()

          2. You can update the object and place it back in the cache, just make sure you wrap it in a transaction, and you use eager locking when reading the object to prevent timeouts due to concurrent writes.

          3. You get a hold of a transaction object by looking up a UserTransaction in JNDI if you are operating in a Java EE environment. You need to make sure Infinispan is configured with a TransactionManagerLookup class though.

          4. This is available. See our FAQs on eager locking.

          • 2. Re: Async Eviction and Correct Cache Usage
            amin59

            I tried following code for issue #1:

             Configuration config = new Configuration();
             GlobalConfiguration globalConfiguration = GlobalConfiguration.getClusteredDefault();
            
             AsyncStoreConfig asyncStoreConfig = new AsyncStoreConfig();
             asyncStoreConfig.setEnabled(true);
            
             CacheStoreConfig cacheStoreConfig = new AccountJdbcCacheStoreConfig();
             cacheStoreConfig.setAsyncStoreConfig(asyncStoreConfig);
             cacheStoreConfig.setPurgeSynchronously(true);
            
             CacheLoaderManagerConfig clmc = new CacheLoaderManagerConfig();
            
             List<CacheLoaderConfig> cacheStoreConfigs = new ArrayList<CacheLoaderConfig>(1);
             cacheStoreConfigs.add(cacheStoreConfig);
            
             clmc.setCacheLoaderConfigs(cacheStoreConfigs);
            
             config.setCacheLoaderManagerConfig(clmc);
            
             CacheManager manager = new DefaultCacheManager(globalConfiguration, config);
            

            But no progress. Still change propagate to DB synchronously. Would you please help me by fixing the above config or sending a fresh sample.

            Thanks in advance.

            • 3. Re: Async Eviction and Correct Cache Usage
              galder.zamarreno

              The configuration looks correct. Having run some quick tests, the put operations on the cache store would occurr asynchronously on a separate thread.

              Enable TRACE and add thread logging information to the logs to verify it on your side.

              • 4. Re: Async Eviction and Correct Cache Usage
                amin59

                Again I added a counter to storeLockSafe() method in my CacheStore class. It's called the same times as cache puts (i.e. 100 puts results in 100 sores) and there is no async activity for aggregating these changes into one store operation.

                Is this because of running without a JTA transaction?

                If there was adequate documentations I would have tried to fix it myself but I all do right now is to read code and guess how to do config.

                Sorry for trouble but it seems I still need more help to fix this issue.

                • 5. Re: Async Eviction and Correct Cache Usage
                  amin59

                  I could not setup async storage via API and switched to config file. Here is my config section for accounts cache.

                  <namedCache name="accounts">
                   <eviction wakeUpInterval="5000" maxEntries="100" strategy="FIFO"/>
                   <expiration lifespan="600" maxIdle="1000"/>
                  
                   <loaders passivation="true" shared="false" preload="true">
                   <loader class="com.magfa.gateway.dao.infinispan.jdbcstore.AccountJdbcCacheStore" fetchPersistentState="true"
                   ignoreModifications="false" purgeOnStartup="true">
                  
                   <properties/>
                   <singletonStore enabled="true" pushStateWhenCoordinator="true" pushStateTimeout="20000"/>
                   <async enabled="true" batchSize="10" threadPoolSize="5"/>
                   </loader>
                   </loaders>
                  </namedCache>
                  


                  Again when I set passivation=false no store operation is performed at all and when set to to true every store happens synchronously.

                  Any help?

                  • 6. Re: Async Eviction and Correct Cache Usage
                    galder.zamarreno

                    Can you please try commenting singletonStore element? That should only be used used when you want a single node in the cluster interacting with the cache. Try without it.

                    If still having issues, attach source code for AccountJdbcCacheStoreConfig, AccountJdbcCacheStore classes.

                    Again, if you enable TRACE and add thread information to each log line (%t in the logging pattern), you should see threads named AsyncStore-* in action if async is working correctly.

                    • 7. Re: Async Eviction and Correct Cache Usage
                      galder.zamarreno

                      For your interest, as mentioned the other day, your configuration, barring issues with AccountJdbcCache* classes looks correct. Here's an example code that uses DummyInMemoryCacheStore:

                      Configuration config = new Configuration();
                      GlobalConfiguration globalConfiguration = GlobalConfiguration.getNonClusteredDefault();
                      AsyncStoreConfig asyncStoreConfig = new AsyncStoreConfig();
                      asyncStoreConfig.setEnabled(true);
                      CacheStoreConfig cacheStoreConfig = new DummyInMemoryCacheStore.Cfg();
                      cacheStoreConfig.setAsyncStoreConfig(asyncStoreConfig);
                      cacheStoreConfig.setPurgeSynchronously(true);
                      CacheLoaderManagerConfig clmc = new CacheLoaderManagerConfig();
                      List<CacheLoaderConfig> cacheStoreConfigs = new ArrayList<CacheLoaderConfig>(1);
                      cacheStoreConfigs.add(cacheStoreConfig);
                      clmc.setCacheLoaderConfigs(cacheStoreConfigs);
                      config.setCacheLoaderManagerConfig(clmc);
                      CacheManager cm = new DefaultCacheManager(globalConfiguration, config);
                      c = cm.getCache();
                      c.put("Galder", "Zamarreno");


                      And here's what the log outputs at TRACE level:

                      2009-06-22 18:23:28,048 TRACE [ReadCommittedEntry] (main) Updating entry (key=Galder removed=false valid=true changed=true created=true value=Zamarreno]
                      2009-06-22 18:23:28,049 TRACE [LockingInterceptor] (main) Releasing lock on [Galder] for owner Thread[main,5,main]
                      2009-06-22 18:23:28,049 TRACE [LockManagerImpl] (main) Attempting to unlock Galder
                      2009-06-22 18:23:28,050 TRACE [AsyncStore] (main) Enqueuing modification Store{storedEntry=ImmortalCacheEntry{key=Galder}}
                      2009-06-22 18:23:28,050 TRACE [AsyncStore] (AsyncStore-0) Calling put(List) with 1 modifications
                      2009-06-22 18:23:28,051 TRACE [AsyncStore] (AsyncStore-0) Checking for modifications
                      2009-06-22 18:23:28,051 TRACE [CacheStoreInterceptor] (main) Stored entry ImmortalCacheEntry{key=Galder} under key Galder


                      From the log, it's quite clear that the modification is enqueued and then, Async-Store-0 thread, a different thread from the main one, actually stores it.

                      • 8. Re: Async Eviction and Correct Cache Usage
                        galder.zamarreno

                        Something else you should try is to verify whether your asynchronous issues is to do with your cache store implementation. To do that, simply switch for example to the FileCacheStore, enable TRACE and see if asynchronous works or not.

                        p.s. I'm unsure why you need a different jdbc cache store implementatipn and what the current jdbc cache lacks that is forcing you to implement your own.

                        • 9. Re: Async Eviction and Correct Cache Usage
                          amin59

                          Dear Galder,

                          Thanks for your help.

                          Before we go ahead and I submit my source codes or test with other cache stores such as FileCacheStore (which I'll try to do ASAP) let me ask one small question.

                          Does the fact that cache store is async merely means that interoperations with databases (or any other data source) is going to be performed by another thread? IMHO this behavior is a consequence of a higher concept which I refer to it as aggregation.

                          I would like Infinispan to work as a write-behind cache. By other means I make hundreds of changes in memory but only the final results (triggered by timer or counter) propagate into database.

                          What I see here is that every modification propagates to database without aggregation. It is right that this happens asynchronously and by another thread but this is not what we usually expect from a write-behind solution.

                          I am looking seeking for a capability similar to Oracle Coherence Cache Store or GigaSpaces EDS in Infinispan.

                          I suppose if I have async store, enable eviction and set passivation to true, Infinispan should work similar but it does not for me.

                          BTW custom JdbcStore is because I have my own scheme and although Infinispan's general jdbc stores are customizable I find it much easier to write one for myself based on my legacy DAO codes rather than customizing SQL expression's of general store.

                          Thanks for you help again. Looking for your answer.

                          • 10. Re: Async Eviction and Correct Cache Usage
                            galder.zamarreno

                            amin59, thanks for explaining your use case much better. I now understand better what your asynch cache store expectations. As you already figured out, our async cache store doesn't do agreegation of the data, it simply queues the modifications and applies them.

                            So, as a result of this, I've created feature request https://jira.jboss.org/jira/browse/ISPN-116 so that this gets addressed asap. The fix version might still change depending on when exactly we schedule it for.

                            • 11. Re: Async Eviction and Correct Cache Usage
                              amin59

                              Galder,

                              To have a quick fix on this, is it possible to switch off this history like behavior in async cache store and config cache to just persist the very last value of every key anytime store method is invoked?

                              IMHO this can be the answer for my requirement.

                              If this is possible with minor code change, I would like to do it with your help.

                              • 12. Re: Async Eviction and Correct Cache Usage
                                galder.zamarreno

                                Hmmmm, I could see something like you suggest working but assuming you only retrieved the latest value of a key in the modification queue. You wouldn't want to navigate every single key in the cache.

                                There's several ways this could be fixed and but rather than discussing it here, it's better to discuss it in the infinispan-dev list and get an agreement on it. So, I'd suggest that you subscribe to it. I'm gonna send an email asap on different solutions.

                                • 13. Re: Async Eviction and Correct Cache Usage
                                  galder.zamarreno

                                  Here's the infinispan-dev thread link http://lists.jboss.org/pipermail/infinispan-dev/2009-June/000375.html

                                  I'd suggest you subscribe to the list via https://lists.jboss.org/mailman/listinfo/infinispan-dev

                                  One thing is for sure that implementing this feature requires some further thought. It's not trivial.