2 Replies Latest reply on Mar 13, 2014 1:34 PM by jugglingcats

    AdvancedCacheLoader.process() method -- purpose and issue for very large store

    jugglingcats

      Hi, I have an implementation of ExternalStore for Mongo that does some object mapping specific to our requirements.

       

      When I started using async mode in our configuration, Infinispan told me the store had to implement AdvancedCacheLoader... so I implemented process() and size() methods no problem based on the SingleFileStore example, but was a bit confused why/when they would be invoked.

       

      Now we're performance testing and one of our caches has 200,000,000 entries or so in Mongo, and the process() method is being called for this cache on start / on joining the cluster. Because we have to interate all entries, this is taking a loooonnng time! In all cases the fetchValue and fetchMetadata parameters are false, so am not sure exactly what it's doing and whether it's necessary.

       

      The large cache in question is in DIST_ASYNC mode and most of the caches use this mode with async enabled. We do have one cache that is REPL_SYNC using the same store implementation with async disabled.

       

      As a workaround I ignore the process call if both fetchValue and fetchMetadata are false, but not sure if this is valid thing to do!! It would be great to get a fuller explanation of the purpose of this method.

       

      Many thanks, Alfie.