Invalidation across a cluster

[manik]

http://jira.jboss.com/jira/browse/JBCACHE-307

This pertains to invalidating (evicting) nodes from remote caches upon modification rather than replicating nodes.

Here is what I have in mind so far with regards to implementing this:

InvalidationInterceptor designs

1. Refactor OptimisticReplicationInterceptor and ReplicationInterceptor to extend an abstract BaseRpcInterceptor (Methods such as replicate(), replicateAsynchronously(), checkResponses()). In future refactorings, this BaseRpcInterceptor could hold the JGroups Channel, etc.

2. InvalidationInterceptor to extend BaseRpcInterceptor

3. InvalidationInterceptor to replace either OptimisticReplicationInterceptor or ReplicationInterceptor in the InterceptorChain if configured.

4. Configured by:

<attribute name="SyncType"></attribute>

Values allowed: REPLICATION (default) or INVALIDATION.

5. Only kicks in when receiving a CRUD method - ignore all other methods.

5.1. If we are not in a tx: broadcast an evict(Fqn).
5.2. Else: register a sync handler. beforeCompletion() will broadcast an evict(Fqn) based on the modification list. Nothing happens in afterCompletion() since an evict() is not a permanent change to remote nodes. Some efficiency can be gained here, by analysing the Fqn's we are evicting, and only calling evict() on the topmost node necessary. E.g., if we need to invalidate /a/b and /a/b/c, only call evict on /a/b.

Comments and thoughts, please?

Cheers,
Manik

[manik]

Changed the config setting to setClusterSyncMode() and getClusterSyncMode().

From the JavaDocs in TreeCache:

 public static final String CLUSTER_SYNC_MODE_REPLICATE = "REPLICATE";
 public static final String CLUSTER_SYNC_MODE_INVALIDATE = "INVALIDATE";

 /**
 * Synchronisation type.
 * REPLICATE (default) replicates modifications across the cluster
 * INVALIDATE sends invalidation messages so remote caches evict() any nodes that have changed. Remote caches
 * would then be forced to look up the nodes in a shared cacheloader if the nodes are needed again.
 */
 protected String clusterSyncMode = CLUSTER_SYNC_MODE_REPLICATE;

[manik]

Ok, I've implemented it in this manner so far, but am not too happy with yet another config parameter.

What does everyone think about incorporating this into the CacheMode setting?

Currently we support the following CacheModes: LOCAL, REPL_SYNC, REPL_ASYNC. Perhaps add INVALIDATION_SYNC and INVALIDATION_ASYNC to this?

Cheers,
Manik

[belaban]

Good idea +1

[ben.wang]

+1 also for me to include it in CacheMode. That makes more sense.

Question though about your invalidation implementation. When node A does a put(fqn, map), if fqn does not exist, should it propagate "replicate" instead of invalidate. Is it the case or not? Or you don't distinguish this?

[manik]

I don't distinguish. I just propagate an evict() whenever a CRUD method is invoked.

[belaban]

I don't think you should replicate ! Maybe the element was evicted, and can be reloaded just-in-time with a backing datastore (JDBCCacheLoader).

[manik]

Final design, as documented in JBossCache/docs/design/InvalidationInterceptor.txt



InvalidationInterceptor
-----------------------

Ref: http://jira.jboss.com/jira/browse/JBCACHE-307
Forum: http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3903158

This interceptor pertains to invalidating (evicting) nodes from remote caches upon modification rather
than replicating nodes.

Here is what I have in mind so far with regards to implementing this:

InvalidationInterceptor designs

1. Refactor OptimisticReplicationInterceptor and ReplicationInterceptor to extend an abstract
BaseRpcInterceptor (Protected methods such as replicateCall(), putCallOnAsyncReplicationQueue(),
checkResponses()). In future refactorings, this BaseRpcInterceptor could hold the JGroups Channel,
etc.

2. InvalidationInterceptor to extend BaseRpcInterceptor

3. InvalidationInterceptor to replace either OptimisticReplicationInterceptor or ReplicationInterceptor
in the InterceptorChain if configured.

4. In addition to the existing cache modes (LOCAL, REPL_SYNC, REPL_ASYNC) we'd also have INVALIDATION_SYNC
and INVALIDATION_ASYNC

5. The interceptor only kicks in when receiving a CRUD method - ignore all other methods.

5.1. If we are not in a tx: broadcast an evict(Fqn).
5.2. Else: register a sync handler. afterCompletion() will broadcast an evict(Fqn) for every
CRUD method in the modification list upon commit. Some efficiency can be gained here,
by analysing the Fqn's we are evicting, and only calling evict() on the topmost node
necessary. E.g., if we need to invalidate /a/b and /a/b/c, only call evict on /a/b.

[manik]

I've updated the documentation in CVS HEAD - docs/TreeCache/en/master.xml

If someone has a spare moment, could you glance over this? I'd appreciate another pair of eyes going over this for me.

Sections 5 and 11 are all I've updated.