Can multiple clients of Infinispan replicated cache share the same persistent file store?

[prateekk.in]

Cross posted at docker - Can multiple clients of Infinispan replicated cache share the same persistent file store? - Stack Overflow

Suppose we have multiple docker containers (each of which has java webapps, so multiple JVMs essentially), each of which are using the same replicated Infinispan cache either in:

• Embedded mode using jgroups for discovery
• Server mode using Docker Hub as Infinispan server, and the clients connecting via hotrod.

In either case, all the cache members/clients have a file-store from which they preload at startup (sample is for server mode, its similar for embedded mode's xml):

Via docker compose, the path inside each container's file store (lets say /var/tmp/server/OUR_CACHE.dat) is bind-mounted to the same file on the docker host.

<paths>
  <path name="cachestore.root" path="/var/tmp"/>
</paths>
...

<local-cache name="OUR_CACHE">

     <expiration lifespan="-1"/>

     <locking isolation="SERIALIZABLE" acquire-timeout="30000" concurrency-level="1000" striping="false"/>

     <file-store relative-to="cachestore.root" path="server" max-entries="-1" purge="false" passivation="false" preload="true" fetch-state="true"/>

     <memory>

          <binary size="100000000" eviction="MEMORY"/>

     </memory>

</local-cache>

My question is that - is the persistence mechanism designed so that multiple replicated cache clients can read/write to/from the same file store without any errors?

My understanding is that in replicated mode, the key values will become eventually consistent across all node's memory. But my goal is to ensure that multiple client containers using the same file-store for persistence and preloading should not adversely affect this replication.

If its not advised to share the same file-store .dat file, then what is the best practice to have a GUID in the filepath for each container's path in the .xml file.
Each docker container's hostname (which is containerId) is unique, but it won't be known before its deployed, so it won't be easy to populate the infinispan_server.xml file with the value for "path" statically.

Thanks,

_Prateek

[galder.zamarreno]

Shared file store can be tricky to handle. Depends on the guarantees/safety that is behind the file sharing. I'm aware of issues when using NFS. File based store is best used by each node locally.

For shared access, cassandra or DB are better suited.

[prateekk.in]

In this case, its the Docker overlayFS drive that is mapping the multiple container paths to the same file path on the docker host's local file system. I wasn't able to find any information on whether the cache's file-store can be used (RW) simultaneously, nor the recommended approach of using embedded/remote infinispan in docker environments. The initial challenge itself is how to generate the mapping (unique file path or unique primary key) to configure infinispan's xml (or the bind mount in docker compose) beforehand - as i will have the docker containerId (=hostname inside container) only after it starts up. Perhaps the approach would be to set the file store programmatically by reading the hostname inside the container and using that with database file store. Thanks.