1 2 Previous Next 18 Replies Latest reply on Feb 18, 2010 12:06 PM by Nicolas Filotto

    How to manage empty data with a CacheLoader?

    Nicolas Filotto Novice

      Hi,

       

      We face a big performance issue because we use a CacheLoader behind a JBC instance that doesn't contain a lot of data but this cache is very frequently accessed and rarely modified. As far as I understand, when we try to get a non existing data from a JBC instance with a CL, JBC does globally the following steps:

      1. It tries to get the data for the local cache

      2. No data cans be found, thus it locks the related fqn

      3. It then gets the data from the backend store

      4. No data cans be found in the backend store, so it has nothing to put into the local cache

      5. It finally releases the fqn

       

      In our case, most of the time there is no data to retrieve from the backend store, so if several threads try to access to the same fqn at the same time we can quickly get a TimeoutExcetion.

       

      For example, if:

      1. The lock timeout has been set to 10 seconds

      2. We need about 1 second to access the backend store

      3. We have 11 threads trying to access the same fqn at the same time

      => The last thread will get a TimeoutException since each thread needs 1 second to do the tasks between step #3 and step #5, so the 10 first threads need 10 seconds to allow the last thread accessing step #3 which won't be possible because the timeout has been set to 10 seconds.

       

      Is there a way to avoid that issue? up to now this is the bottleneck of our entire application.

       

      The only idea that I have so far is to put a fake data into the local cache representing some how "no data", just to avoid accessing the backend store anytime we want to access the value from the cache. But I still have 2 questions which are:

       

      1. What is the best way to implement such mechanism (i.e. loading fake data)?

      2. The previous workaround described below, only solve the issue for the 2nd access to the data, the issue remains for the 1st access when the data representing "no data" has not yet been loaded. Is there a way to solve also this part of the problem?

       

      Thank you in advance for your help,

      Best Regards,

      Nicolas Filotto

        • 1. Re: How to manage empty data with a CacheLoader?
          Manik Surtani Master

          This is a common problem with cache stores.  If stuff isn't in memory there is every chance it is stored elsewhere and this must be checked.

           

          Optimising the cache store will help to some degree, but if most of your threads result in cache misses, then you should rethink why you use a cache store (or even a cache!) in the first place. 

           

          Starting with optimisation, the JDBC cache store is one of the slowest, due to remote network connections and all sorts of bottlenecks in databases, combined with the fact that most databases need to be tuned extensively to start performing.  One recommendation is to consider other forms of cache storage.  BDBJE is a good engine, fast and powerful.

           

          If you have to live with a JDBC backend, I recommend taking a look at Infinispan's JDBC cache store - a complete rewrite from the JBoss Cache one, and with some interesting optimisations in place that will help perform better.  Note that Infinispan's JDBC cache store is not compatible with JBoss Cache, but you could enhance JBoss Cache's implementation using similar techniques.

           

          But both of the above techniques just improve the performance of the cache store and not really solve your problem.  Regarding your solution of using a "dummy entry", this does make sense provided you have a known and finite set of keys.  But if you know this set of keys and it is finite, then why do you use a cache store, since if it is to deal with persisting evicted objects, then you would still always have to check the store for existence of the object.

           

          Cheers

          Manik

          • 2. Re: How to manage empty data with a CacheLoader?
            Manik Surtani Master

            One approach you could use - and this would be a hack - is that *if* you are aware of the keys/nodes used and know that a get() will result in a cache miss, you could pass in a flag to bypass the cache loading from the store altogether. 

             

            There is a similar option in place to suppress persistence, and you could use this same flag in a subclass of the CacheLoaderInterceptor to bypass loading if this option is set, and then use your subclass interceptor in place of the CacheLoaderInterceptor.

            • 3. Re: How to manage empty data with a CacheLoader?
              Nicolas Filotto Novice

              Hi Manik,

               

              Thank you for your clear answer, actually we use JBC with a CacheLoader in that particular use case (which is in fact to implement the Lock Manager of our application) for the following requirements that are fulfilled by JBC with a CL :

              1. The data of the locks must be replicated over the cluster
              2. The data of the locks must not be modified at the same time to ensure their consistency (i.e 2 different users cannot add a lock on the same resource)
              3. The data of the locks must be persisted

               

              >> But if you know this set of keys and it is finite, then why do you use a cache store, since if it is to deal with persisting evicted objects, then you would still always have to check the store for existence of the object.

               

              << That is right, but all the data is loaded at startup, so we know that if the data is not in the local cache, it means that it doesn't exist in the database since the only way to add data into the database is to go though the JBC instance and its CacheLoader.

              • 4. Re: How to manage empty data with a CacheLoader?
                Nicolas Filotto Novice

                Yes I saw that, I already tried something like this:

                {code}

                cache.getInvocationContext().getOptionOverrides().setSuppressPersistence(true);

                cache.get(fqn, key);

                {code}

                 

                But it doesn't have any effect, I guess that it is more for the put method

                • 5. Re: How to manage empty data with a CacheLoader?
                  Manik Surtani Master
                  Are you using eviction, by any chance?
                  • 6. Re: How to manage empty data with a CacheLoader?
                    Nicolas Filotto Novice
                    I don't understand how it is related, could you please describe more what you have in mind?
                    • 7. Re: How to manage empty data with a CacheLoader?
                      Nicolas Filotto Novice

                      To solve my issue I had to write a decorator on the top of the current CacheLoader in order to rewrite the methods corresponding to read operations. Since I load everything at startup, I can by-pass the CacheLoader call when the cache is started.

                       

                      See below the code:

                      {code:java}

                      public class ControllerCacheLoader implements CacheLoader

                      {

                         /**

                          * The nested cache loader

                          */

                         private final CacheLoader cl;

                      ...

                         /**

                          * @see org.jboss.cache.loader.CacheLoader#exists(org.jboss.cache.Fqn)

                          */

                         public boolean exists(Fqn name) throws Exception

                         {

                            if (cache.getCacheStatus() == CacheStatus.STARTING)

                            {

                               // Before calling the nested cache loader we first check if the data exists in the local cache

                               // in order to prevent multiple call to the cache store       

                               NodeSPI<?, ?> node = cache.peek(name, false);

                               if (node != null)

                               {

                                  // The node already exists in the local cache, so we return true

                                  return true;          

                               }

                               else

                               {

                                  // The node doesn't exist in the local cache, so we need to check through the nested

                                  // cache loader

                                  return cl.exists(name);       

                               }

                            }

                            // All the data is loaded at startup, so no need to call the nested cache loader for another

                            // cache status other than CacheStatus.STARTING

                            return false;

                         }

                       

                         /**

                          * @see org.jboss.cache.loader.CacheLoader#get(org.jboss.cache.Fqn)

                          */

                         public Map<Object, Object> get(Fqn name) throws Exception

                         {

                            if (cache.getCacheStatus() == CacheStatus.STARTING)

                            {

                               // Before calling the nested cache loader we first check if the data exists in the local cache

                               // in order to prevent multiple call to the cache store                

                               NodeSPI node = cache.peek(name, false);

                               if (node != null)

                               {

                                  // The node already exists in the local cache, so we return the corresponding data

                                  return node.getDataDirect();          

                               }

                               else

                               {

                                  // The node doesn't exist in the local cache, so we need to check through the nested

                                  // cache loader          

                                  return cl.get(name);       

                               }

                            }

                            // All the data is loaded at startup, so no need to call the nested cache loader for another

                            // cache status other than CacheStatus.STARTING

                            return null;

                         }

                       

                         /**

                          * @see org.jboss.cache.loader.CacheLoader#getChildrenNames(org.jboss.cache.Fqn)

                          */

                         public Set<?> getChildrenNames(Fqn fqn) throws Exception

                         {

                            if (cache.getCacheStatus() == CacheStatus.STARTING)

                            {

                               // Try to get the list of children name from the nested cache loader

                               return cl.getChildrenNames(fqn);       

                            }

                            // All the data is loaded at startup, so no need to call the nested cache loader for another

                            // cache status other than CacheStatus.STARTING

                           return null;

                         }

                      ...

                      {code}

                       

                      See below the config

                      {code:xml}

                      <?xml version="1.0" encoding="UTF-8"?>

                      <jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:jboss:jbosscache-core:config:3.1">

                      ..

                         <loaders passivation="false" shared="true">

                            <!-- All the data of the JCR locks needs to be loaded at startup -->

                        <preload>

                           <node fqn="/" />

                        </preload>

                      ..

                      {code}

                      • 8. Re: How to manage empty data with a CacheLoader?
                        Manik Surtani Master

                        If you are *not* using eviction, then stuff would never *not* be in memory and in the cache loader instead.  So you could effectively subclass the CLI to be a no-op for GET, CONTAINS, etc.

                        • 9. Re: How to manage empty data with a CacheLoader?
                          Manik Surtani Master

                          What you have here makes no sense.  CacheLoader.exists() and get() will never be called if the node is in memory already.  Have a look at the CacheLoaderInterceptor for details.

                           

                          Like I said, what makes more sense is, if you never use eviction and can guarantee that what is in memory will never be out if sync with what is in the cache loader, then subclass the CLI and override the methods that load stuff from the cache loader.

                          • 10. Re: How to manage empty data with a CacheLoader?
                            Nicolas Filotto Novice

                            >> What you have here makes no sense.  CacheLoader.exists() and get() will never be called if the node is in memory already.  Have a look at the CacheLoaderInterceptor for details

                             

                            << Yes, but CacheLoader.exists() and get() are always called when the node is missing into the local cache which is the case 90% of the time in my usecase that is why I did that

                             

                            >> Like I said, what makes more sense is, if you never use eviction and can guarantee that what is in memory will never be out if sync with what is in the cache loader, then subclass the CLI and override the methods that load stuff from the cache loader.

                             

                            << Yes, but I don't know which CacheLoader implementation will be used by the end-user i.e. FileCacheLoader, JDBCCacheLoader.. That is why I had to use a decorator instead in order to propose a solution that works with all the CacheLoader implementations.

                            • 11. Re: How to manage empty data with a CacheLoader?
                              Galder Zamarreño Master
                              The CLI (as in  CacheLoaderInterceptor) that Manik refers to always gets used regardless of the cache loader implementation. So your rationale to use a cache loader decorator vs subclassing CLI does not stand. CLI does not stand for CacheLoader implementation.
                              • 12. Re: How to manage empty data with a CacheLoader?
                                Nicolas Filotto Novice
                                Thanks Galder for this very helpful clarification, indeed extending the CacheLoaderInterceptor can be an interesting approach but (for me) only to allow me to inject properly my decorator because the decorator only relies on a public API so it is less risky to use it in order to easily ensure forward compatibility. Up to now, my way to inject my decorator could be potentially incompatible with the next versions of JBC so I'm very interested in knowing the "official" way to replace the current CacheLoaderInterceptor by my own sub-class. I have seen in the javadoc that we have method such as cache.removeInterceptor and cache.addInterceptor, but how to ensure that our own CacheLoaderInterceptor implementation will be at the exact same position knowing that the position is critical in JBC?
                                • 13. Re: How to manage empty data with a CacheLoader?
                                  Nicolas Filotto Novice

                                  I tried several things to change the CacheLoaderInterceptor with my own version with no success, such as:

                                   

                                  {code:java}

                                  ControllerCacheLoaderInterceptor cci = new ControllerCacheLoaderInterceptor();

                                  // replace the default CacheLoaderInterceptor by the new one in the interceptor chain

                                  cache.addInterceptor(cci, CacheLoaderInterceptor.class);

                                  cache.removeInterceptor(CacheLoaderInterceptor.class);

                                  // replace the default CacheLoaderInterceptor by the new one in the component registry

                                  ((CacheSPI<Serializable, Object>)cache.cache).getComponentRegistry().registerComponent(cci, CacheLoaderInterceptor.class);

                                  {code}

                                   

                                  I tried to add those lines of code before cache.create() and/or before cache.start() with no success.

                                  • 14. Re: How to manage empty data with a CacheLoader?
                                    Mircea Markus Master
                                    I do generally agree with this, but JBC is in maintenance mode now, and infinispan is here to take its place, so the chance to do API chances is virtually 0. 
                                    1 2 Previous Next