When should i use CacheMode Invalidation

[hackland]

Hello everybody,

i was searching for an example for the use of CacheMode "Invalidation". I would like to write a documentation of Infinispan in the german language.

So i would like to explain the CacheMode Invalidation with a good example. I read the dokumentation but i'm not realy sure i understand it correct.

Thank you for your help.

Best regards

Andy

[mgencur]

The Invalidation mode becomes handy when you have a read-heavy scenario. It means that you mostly do read operations and rarely write operations.

So it's useful when Infinispan is used as an optimisation when reading from a permanent storage (like DB) and there are more Infinispan nodes. ISPN caches values read from the permanent store and only invalidates values in other nodes when a write operation is performed. As is written in the documentation, this mode minimizes network traffic as invalidation messages are small. Furthermore, data is loaded from the permanent store to the cache lazily - only when needed.

Is it more clear?

[hackland]

Hi Martin,

thank you for your helpfull answer. It makes more clear.

Is the following detailed example correct?

----------- Example  ----------------------

I have a DB as a permananet storages for my data in Cluster "C1".

The data is primary hold in the DB. 

I have two Infinispan nodes <A> <B>  that represtens together a cache with the name "Cache1".

At the beginning of the cluster, data is only in the DB an the nodes are empty.

Now I get many read request randomly onto the two nodes.

1. For example on node <A> I get read requests for data (x),(y) and (z).

    -> Node <A> read the data from DB and save (x),(y),(z) local.

2. On node B i get read requests for (a) and (z).

    -> Node <B> read the data from DB and save (a) and (z) local

3. So if I get later an read request on Node <A> for data (z).

    -> Node <A> don't need to read (z) from the DB because it is local saved.

4. Then I get a write request (update data (z)) on Node <A> for data (z)

    -> Node <A> change the value (z) local and in the DB (or is it first cached in DB and then local?)

           and send an Invalide message to all the other nodes (multicast) that represents the "Cache1"

    -> Node <B> set data (z) as invalidate.

5. Then I get a read request on node "B" for data (z)

    -> Node <B> get the new version for data (z) from the DB and save the new Version local.

------------ End Example --------------------------

If this example is correct:

-> The System is at the beginning slow for read request!

    If so, is it possible, respectively recommended that I load data at the beginning into the nodes (initial state transfer)?

Thank you for your help!

[mgencur]

You described it exactly how I understand it

(or is it first cached in DB and then local?)  

I think it's first updated locally in the cache and then stored in the DB but I'm not sure about it. Hovewer, it shouldn't matter to you.

If so, is it possible, respectively recommended that I load data at the beginning into the nodes (initial state transfer)?

It's up to you to decide whether it is good to preload the data into the cache or not. The respective attribute is called "preload" and can be set on <loaders> element. The startup of the cache will be certainly slower but requests for data will be fast from the very beginning.

[darrellburgan]

hackland wrote:

1. For example on node <A> I get read requests for data (x),(y) and (z).

    -> Node <A> read the data from DB and save (x),(y),(z) local.

2. On node B i get read requests for (a) and (z).

    -> Node <B> read the data from DB and save (a) and (z) local

One nuance we encountered is that to get the behavior you described, you must use putForExternalRead() not put() to save data to the local cache. If you use put() then in step 2 when you put (z) to the local cache on B, (z) is invalidated from the local cache on A. It was a huge difference to our system when we started using putForExternalRead(), because then it began to behave exactly as you described.

[mgencur]

Darell,

when you use putForExternalRead() instead od put(), you can end up with different key/value pairs in individual nodes. Then when you read x from <A>, it's different then x at <B>. I thought putForExternalRead() is mainly used internally. When a user requests a key/value from a cache and it is only in the cache store, Infinispan internally get the value from the cache store and uses putForExternalRead() for storing it into local cache - so that it does not send invalidation message to other nodes in this case. Invalidation in this case would not make sense.

[darrellburgan]

Martin Gencur wrote:

when you use putForExternalRead() instead od put(), you can end up with different key/value pairs in individual nodes. Then when you read x from <A>, it's different then x at <B>. I thought putForExternalRead() is mainly used internally. When a user requests a key/value from a cache and it is only in the cache store, Infinispan internally get the value from the cache store and uses putForExternalRead() for storing it into local cache - so that it does not send invalidation message to other nodes in this case. Invalidation in this case would not make sense.

Thanks for the help Martin. So the way our old custom ORM works is as follows, with regard to caching (at a high level). Note that all our caches are invalidation caches, there is no transaction manager in our old system, and all database writes happen through the custom ORM.

// assume all reads of widgets go through this method

public Widget readWidget(int widgetId) {

     Widget w = cache.get(widgetId);

     if (w == null) {

          w = (call ORM to get the widget for widgetId ...);

          cache.putForExternalRead(widgetId, w);

     }

     return w;

}

// assume ALL updates to widgets go through this method

public void save(Widget w) {

     cache.remove(w.getWidgetId());

     (call ORM to save the widget to the database);

}

// assume ALL deletes of widgets go through this method

public void delete(Widget w) {

     cache.remove(w.getWidgetId());

     (call ORM to delete the widget from the database);

}

Here's my understanding of how this relates to cache inconsistency across nodes: yeah it is possible that different nodes will have different values for a given widget id X. But the eventually consistent state is that a particular node either has an up to date version of X in their cache, or they have nothing at all for X. There will be transient states where one node has an obsolete version of X, but only until the invalidation message is received and it 'catches up' to the node that invalidated it. As long as this latency between the time node A invalidates an entry and node B invalidates the same entry is short enough, there should be no issue. I agree that during that time it is in an inconsistent state, but I'm willing to live with that tradeoff as long as the latency is measured in millis not seconds.

Am I missing something?

[mgencur]

Darrell,

now I understand why putForExternalRead is an ideal choice for you ....from the code. It seems to me that it's safe. You basically mimic internal Infinispan behavior - by calling putForExternalRead when the entry is not in the cache. The remove operation sends invalidation messages anyway so it's safe IMO.

[darrellburgan]

Martin Gencur wrote:

Darrell,

now I understand why putForExternalRead is an ideal choice for you ....from the code. It seems to me that it's safe. You basically mimic internal Infinispan behavior - by calling putForExternalRead when the entry is not in the cache. The remove operation sends invalidation messages anyway so it's safe IMO.

Thanks - good to know we're not way out in left field ...