Write Behind implementation in Infinispan

[prashant.thakur]

We are trying to have Infinispan configured for a write behind to Oracle DB using JDBC.

Need to understand the logic or sequence of events its done as things are not clear from documentation.

We would like to store a running balance value (float value) for key which can be updated by multiple clients. In write behind we understand that the values are store in queue and then written to the CacheStore (Single DB in our case) at later stage.

The points we need to understand are

1) Are the updates written by a single node of cluster or multiple nodes ? SingletonStoreConfiguration class I saw but not sure how we can configure the same.

2) Are the update queues maintained across cluster the same or are for each node individually ?

3) If its written by one node and then one node goes down will the other node be able to write the changes to DB. How does Infinispan guarantees that information is not lost in case of failures.

Any  pointers to relevant classes would be useful .

Thanks,

Prashant

[sannegrinovero]

Hi,

I'm assuming you talks about shared cachestores.

the queue is per-node : it's not replicated and a crash of the individual node will cause the operations to not be applied.

Still, the data is consistent in the cluster (in-memory) so reading an existing value will return you the correct value. Reading of a deletion however might trigger a CacheStore load, and retrieve a stale value.

The operation is performed by the primary owner of each key (the first one of the list of owners to which the key would be stored).

[prashant.thakur]

Thanks Sanne for your reply.

I had one more configuration related doubt. We did configuration for Singleton and Async combination but somehow its not working . It seems something is missing.

Can you please have a quick look at the same ?

What should be the Cache mode selected for this behaviour CacheMode.DIST_SYNC seems most appropriate as this would ensure sync between all Inmemory Instances but this probably is blocking Asyc writes to the DB Store This I am expecting from the function below .

public boolean isSynchronous() {

      return this == REPL_SYNC || this == DIST_SYNC || this == INVALIDATION_SYNC || this == LOCAL;

   }

Our complete configuration from code is as below Are we missing something as when we remove the async part the write operation is called but adding async parameters is skipping the calls to Persistent store

We are not reaching this function when async part is added  public void write(MarshalledEntry entry) in JdbcStringBasedStore

The code to create configuration is as below.

ConfigurationBuilder builder = new ConfigurationBuilder();

          builder.clustering().cacheMode(CacheMode.DIST_SYNC).hash().numOwners(3).persistence().addStore(JdbcStringBasedStoreConfigurationBuilder.class)

             .fetchPersistentState(false)

             .ignoreModifications(false)

             .purgeOnStartup(false)        

             .table()

                .dropOnExit(false)

                .createOnStart(false)

               ....

             .connectionPool()

                .connectionUrl("jdbc:oracle:thin:@127.0.0.1:1521:orcl11g")

                .username("db_user")

                .password("db_password")

                .driverClass("oracle.jdbc.driver.OracleDriver")

                .singleton().enabled(true).pushStateWhenCoordinator(false)

                .async().enabled(true).modificationQueueSize(2000).threadPoolSize(23)

                ;

For deletion part :

Reading of a deletion however might trigger a CacheStore load, and retrieve a stale value.

From documentation I have a different understanding which says that even if entries are deleted a copy of key would be kept in Memory which shouldn't trigger reading stale information.

Do you mean to say this might happen when this key is evicted we will have this issue. How do we persist this key deletion information ?

6.5.4. Tombstones

To deal with deletions of entries, tombstones will be maintained as null entries that have been deleted, so that version information of the deleted entry can be maintained and write skews can still be detected. However this is an expensive thing to do, and as such, is a configuration option, disabled by default. Further, tombstones will follow a strict lifespan and will be cleared from the system after a specific amount of time.