-
1. Re: Transactional behavior when using CACHE_MODE_LOCAL flag
rvansa Feb 16, 2017 4:14 AM (in response to jose.damaso)The flag should be honored, and if there's no non-local operation in the transaction, it should not go to remote nodes. How have you observed that the transaction goes remote? And what version do you use?
-
2. Re: Transactional behavior when using CACHE_MODE_LOCAL flag
jose.damaso Feb 16, 2017 5:09 AM (in response to rvansa)I am load testing my application (only read scenarios, but both nodes are
trying to load the same entries on their cache concurrently) and noticed a
lot of replication timeout errors and poor performance. As soon as I change
the cache to a local cache or disable transactions, the errors disappear
and performance goes back to normal.
Some technical details (I am using Infinispan 8.2.6 - infinispan-core dependency):
Infinispan configuration (the "catalog.version" cache is not being used in my load test scenario):
<?xml version="1.0" encoding="UTF-8"?> <infinispan xmlns="urn:infinispan:config:8.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:infinispan:config:8.2 http://infinispan.org/schemas/infinispan-config-8.2.xsd"> <jgroups> <stack-file name="jgroups-udp" path="jgroups-udp.xml"/> </jgroups> <cache-container default-cache="default"> <transport stack="jgroups-udp"/> <jmx domain="sds.infinispan"/> <invalidation-cache name="default"> <transaction mode="NON_XA"/> <store-as-binary/> </invalidation-cache> <distributed-cache name="catalog.version"/> <invalidation-cache name="catalog.materializations"/> <invalidation-cache name="catalog.tariff.subscription.specificity"/> <invalidation-cache name="oidc.clients"/> <invalidation-cache name="preference.preferences"/> <invalidation-cache name="preference.values"/> <invalidation-cache name="sap.sysConf"/> </cache-container> </infinispan>
Put (load) operation (using a cache wrapper to have a friendlier interface):
package pt.link.sc.sds.cache; import org.infinispan.cache.impl.AbstractDelegatingCache; import java.util.function.Supplier; import static org.infinispan.context.Flag.*; public class Cache<K, V> extends AbstractDelegatingCache<K, V> { public Cache(org.infinispan.Cache<K, V> cache) { super(cache); } public V getOrLoad(K key, Supplier<V> loader) { V value = get(key); if (value == null) { value = loader.get(); getAdvancedCache() .withFlags(CACHE_MODE_LOCAL, FAIL_SILENTLY, ZERO_LOCK_ACQUISITION_TIMEOUT) .put(key, value); } return value; } }
-
3. Re: Transactional behavior when using CACHE_MODE_LOCAL flag
rvansa Feb 16, 2017 10:02 AM (in response to jose.damaso)It seems that you're right, care to create a JIRA and link to this forum thread? It seems that the PrepareCommand is sent with single modification: InvalidateCommand, with empty set of keys.
PS: you can even contribute a fix
Here is a test (in o.i.invalidation.BaseInvalidationTest) that reproduces it:
public void testLocalModeTxPutDoesNotGoRemote() throws Exception { AdvancedCache cache1 = cache(0,"invalidationTx").getAdvancedCache(); AdvancedCache cache2 = cache(1,"invalidationTx").getAdvancedCache(); RpcManager originalRpcManager = TestingUtil.extractComponent(cache1, RpcManager.class); TestingUtil.replaceComponent(cache1, RpcManager.class, new AbstractControlledRpcManager(originalRpcManager) { @Override protected Object beforeInvokeRemotely(ReplicableCommand command) { throw new IllegalStateException("Should not invoke remote operations"); } }, true); TransactionManager tm1 = TestingUtil.getTransactionManager(cache1); tm1.begin(); cache1.withFlags(CACHE_MODE_LOCAL).put("key", "value"); tm1.commit(); assertEquals("value", cache1.get("key")); assertNull(cache2.get("key")); }
-
4. Re: Transactional behavior when using CACHE_MODE_LOCAL flag
william.burns Feb 16, 2017 10:28 AM (in response to rvansa)I personally think the way this behaves in general should be reevaluated. To me an Invalidation cache has only purpose and that is to send InvalidateCommand messages to the cluster. No other commands should be sent. In this case the lock command, prepare, commit, rollback etc should never be sent across. Maybe people envisioned this being able to be used to to control locking behavior when using a tx? Unfortunately without a primary owner we will run into deadlocks like Infinispan DIST used to and exactly what Jose ran into.
The InvalidateCommand for a Tx shouldn't be sending an empty set of keys either like in Radim's test
-
5. Re: Transactional behavior when using CACHE_MODE_LOCAL flag
rvansa Feb 16, 2017 11:40 AM (in response to william.burns)I agree with Will - the locks seemingly don't work with invalidation cache. But just broadcasting invalidate command is not enough: usually you want to start invalidating the entry during prepare phase, block (ignore) any writes to that during the DB commit and stop invalidating after commit. XA transactions can't work well, since you can't set up the order of XA resources during prepare/commit and the invalidation has to "wrap" the DB commit.
This comes up pretty often, and I have an itch that we should accomodate some generic tier (similar cache mode, but likely with different API) that will cover both ORM 2LC use case, as well as other uses as cache (e.g. what Keycloak uses).