8 Replies Latest reply on Oct 26, 2005 8:20 AM by Manik Surtani

    Invalidation across a cluster

    Manik Surtani Master

      http://jira.jboss.com/jira/browse/JBCACHE-307

      This pertains to invalidating (evicting) nodes from remote caches upon modification rather than replicating nodes.

      Here is what I have in mind so far with regards to implementing this:

      InvalidationInterceptor designs

      1. Refactor OptimisticReplicationInterceptor and ReplicationInterceptor to extend an abstract BaseRpcInterceptor (Methods such as replicate(), replicateAsynchronously(), checkResponses()). In future refactorings, this BaseRpcInterceptor could hold the JGroups Channel, etc.

      2. InvalidationInterceptor to extend BaseRpcInterceptor

      3. InvalidationInterceptor to replace either OptimisticReplicationInterceptor or ReplicationInterceptor in the InterceptorChain if configured.

      4. Configured by:

      <attribute name="SyncType"></attribute>
      

      Values allowed: REPLICATION (default) or INVALIDATION.

      5. Only kicks in when receiving a CRUD method - ignore all other methods.

      5.1. If we are not in a tx: broadcast an evict(Fqn).
      5.2. Else: register a sync handler. beforeCompletion() will broadcast an evict(Fqn) based on the modification list. Nothing happens in afterCompletion() since an evict() is not a permanent change to remote nodes. Some efficiency can be gained here, by analysing the Fqn's we are evicting, and only calling evict() on the topmost node necessary. E.g., if we need to invalidate /a/b and /a/b/c, only call evict on /a/b.

      Comments and thoughts, please?

      Cheers,
      Manik


        • 1. Re: Invalidation across a cluster
          Manik Surtani Master

          Changed the config setting to setClusterSyncMode() and getClusterSyncMode().

          From the JavaDocs in TreeCache:

           public static final String CLUSTER_SYNC_MODE_REPLICATE = "REPLICATE";
           public static final String CLUSTER_SYNC_MODE_INVALIDATE = "INVALIDATE";
          
           /**
           * Synchronisation type.
           * REPLICATE (default) replicates modifications across the cluster
           * INVALIDATE sends invalidation messages so remote caches evict() any nodes that have changed. Remote caches
           * would then be forced to look up the nodes in a shared cacheloader if the nodes are needed again.
           */
           protected String clusterSyncMode = CLUSTER_SYNC_MODE_REPLICATE;
          
          



          • 2. Re: Invalidation across a cluster
            Manik Surtani Master

            Ok, I've implemented it in this manner so far, but am not too happy with yet another config parameter.

            What does everyone think about incorporating this into the CacheMode setting?

            Currently we support the following CacheModes: LOCAL, REPL_SYNC, REPL_ASYNC. Perhaps add INVALIDATION_SYNC and INVALIDATION_ASYNC to this?

            Cheers,
            Manik

            • 4. Re: Invalidation across a cluster
              Ben Wang Master

              +1 also for me to include it in CacheMode. That makes more sense.

              Question though about your invalidation implementation. When node A does a put(fqn, map), if fqn does not exist, should it propagate "replicate" instead of invalidate. Is it the case or not? Or you don't distinguish this?

              • 5. Re: Invalidation across a cluster
                Manik Surtani Master

                I don't distinguish. I just propagate an evict() whenever a CRUD method is invoked.

                • 6. Re: Invalidation across a cluster
                  Bela Ban Master

                  I don't think you should replicate ! Maybe the element was evicted, and can be reloaded just-in-time with a backing datastore (JDBCCacheLoader).

                  • 7. Re: Invalidation across a cluster
                    Manik Surtani Master

                    Final design, as documented in JBossCache/docs/design/InvalidationInterceptor.txt



                    InvalidationInterceptor
                    -----------------------

                    Ref: http://jira.jboss.com/jira/browse/JBCACHE-307
                    Forum: http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3903158

                    This interceptor pertains to invalidating (evicting) nodes from remote caches upon modification rather
                    than replicating nodes.

                    Here is what I have in mind so far with regards to implementing this:

                    InvalidationInterceptor designs

                    1. Refactor OptimisticReplicationInterceptor and ReplicationInterceptor to extend an abstract
                    BaseRpcInterceptor (Protected methods such as replicateCall(), putCallOnAsyncReplicationQueue(),
                    checkResponses()). In future refactorings, this BaseRpcInterceptor could hold the JGroups Channel,
                    etc.

                    2. InvalidationInterceptor to extend BaseRpcInterceptor

                    3. InvalidationInterceptor to replace either OptimisticReplicationInterceptor or ReplicationInterceptor
                    in the InterceptorChain if configured.

                    4. In addition to the existing cache modes (LOCAL, REPL_SYNC, REPL_ASYNC) we'd also have INVALIDATION_SYNC
                    and INVALIDATION_ASYNC

                    5. The interceptor only kicks in when receiving a CRUD method - ignore all other methods.

                    5.1. If we are not in a tx: broadcast an evict(Fqn).
                    5.2. Else: register a sync handler. afterCompletion() will broadcast an evict(Fqn) for every
                    CRUD method in the modification list upon commit. Some efficiency can be gained here,
                    by analysing the Fqn's we are evicting, and only calling evict() on the topmost node
                    necessary. E.g., if we need to invalidate /a/b and /a/b/c, only call evict on /a/b.

                    • 8. Re: Invalidation across a cluster
                      Manik Surtani Master

                      I've updated the documentation in CVS HEAD - docs/TreeCache/en/master.xml

                      If someone has a spare moment, could you glance over this? I'd appreciate another pair of eyes going over this for me.

                      Sections 5 and 11 are all I've updated.