Peculiar behavior of invalidation-cache?

[wwang2016]

Hi,

I was investigating the invalidation-cache setting for entity in wildfly 10.1.0, and found some peculiar behavior. I am not sure if this is by design or if I need to make any configuration change.

I used the out-of-box standalone-full-ha.xml to start up the two wildfly instances

I have a cluster of two wildfly 10.1.0.Final instance (w#1 and w#2). Both instances connect to the same database.

user #1 log on to w#1

user #2 log on to w#2

user #1 search for a list of hotels, and I can see the cacheable query was printed out in the console. User #1 continue to view the hotel (id = 1). No query was printed out. Note, this is using EntityManager.find(), and it is expected that the object was cached, so no query was printed out.

user #2  did the same things, and the observation was same.

User #1 edited the hotel object (id=1), the update statement was printed out. user #1 go back to search page and revisited the same hotel (id=1), no query was printed out. This is good since the latest update statement cached the result.

User #2 visited the hotel object, and a query to find a single object was printed out. This was expected since the cache for the object was invalidated. However, when user #2 revisited the same object, the query got printed out again, and this happened until after a period of time (probably the configured expiration).

The behavior indicates that the wildfly instance which actually made the modification of an object was able to cache the new data, but the wildfly instance (of the cluster) which received the signal to invalidate the cache did just that: invalidate the cache, but not create new cache for the latest data.

Is this the intended design by infinispan? or if there is any other configuration I need to make?

Thanks

Wayne

[rvansa]

This is not the expected behaviour; after reading the entity from DB it should get cached. In fact the invalidation in 2LC is two-phase (as opposed to vanilla Infinispan invalidation mode) to prevent you from getting stale results, so this sounds like the 'unlock' part did not finish correctly. There are also some additional checks that depend on wall-clock time, though; could you check that the wall-clock time on those machines is in sync?

The default WildFly configuration uses transactional caches wher Infinispan registers as JTA synchronzation (that's the NON_XA). 2LC now supports no-transactional caches, too, so you can try switching that to NONE.

[wwang2016]

Hi Radim,

I started the two instances in the same machine (with different sets of port number), so the time is sync.

I indeed change the out-of-box configuration from NON_XA to NONE, and I saw the instance that receives the signal to invalidate the cache was able to retrieve the latest data, and cached it since it did not print out the query twice.

Just out of curiosity, why did the setting of NON_XA broke the proper cache management?

Thanks,

Wayne

[rvansa]

No idea what's wrong in your case. There may be a bug that's avoided in non-transactional scenario, though the default is tested in the same way.

[vdzhuvinov]

This may be related to the following issue I posted a while ago:

[ISPN-7104] Cache remove fails to return value in invalidation mode with cache store - JBoss Issue Tracker

The way we worked around it is to always do a get(key) before a write operation (with a cache store enabled).

[wwang2016]

Hi Vladimir,

Did you describe a scenario of working on an invalidation cache directly? I was describing the default configuration of entity cache being invalidation-cache, and the instance that receives a signal to invalidate the cache (e.g. update an object in another instance) did invalidate the cache, but did not build the cache again with the latest data.

Setting NON_XA  (out-of-box configuration in wildfly 10.1.0.Final) to NONE fixed the issue.

[vdzhuvinov]

Yes, in our app we deal with Infinispan and its caches directly, and certainly with no XA. Just wanted to mention that the Cache methods in invalidation mode can return incorrect results, which could be a cause.