Is using NFS "bad"? If so, what's the best alternative?

[gsaxena888]

I'm a newbie (but a touch confused)...if I want to persist data to a storage system and I am not tied to any legacy solution (ie I can chose the simplest one to maintain that's still reasonably performant), why can't I use nfs? (The user guide says that nfs should be avoided due to locking issues, but then what's the alternative -- do I need to actually have a rdbms? (And if nfs is not meant to be used, then is file store only useful for single node deployments?) Ignite uses a simple file that can, I think, be on nfs for persistence, so I thought Infinispan could potentially do the same.) Note: In case it helps, I do NOT need transactions support for 99% or even possibly 100% of cases.... (I intend to deploy Infinispan on Google Cloud or maybe AWS, and so using NFS as the persistence backing layer seemed to make the most sense).

Also, on a related note, the "soft index" is considered beta, but are there any known "cons" of using it? Is the performance comparable/better than the original file store? (Is it as stable -- and if so, I assume it's been in use by a reasonable number of users under real conditions for a certain period of time?)

[galder.zamarreno]

If you want to use file based persistence, it's best to use it local to each node as opposed to as a shared cache store. This works just fine. In a cloud environment, you probably want to direct the location to a volume that survives restarts, like a persistent volume in Kubernetes.

rvansa can probably best respond to the status of soft index cache store. I've not seen that many users using it in the community, so might be a bit less mature that other options.

[rvansa]

First of all, the fact that NFS is shared does not mean that you can share the location for any file-based cache store between nodes. Each node must have mutually exclusive access to its data.

The first question is why you want to persist the data. If you just need to offload it from memory (and don't care much about the durability since you have more replicas), you can probably use single file store even on NFS. There are issues when the process/machine writing to the file crashes, when the lock is held and therefore if you restart the node, you might need to manually break the lock.

If you really want the data persist even if the node crashes, My guess is that there's good chance that both NFS-based and filesystem-based single-file-store file would get corrupt and would be unreadable, as the write access has random pattern and part of operation might not be flushed. That gets us to the soft index file store - this guy uses append-only logs for the data so there's a better chance for recovery, though I have not tested that and without tooling it might be troublesome.

Regarding SIFS maturity - this was marked experimental since it was a substantially sized piece of code added by single contributor at once, but I think it stable now - the only unresolved issue I recall were startup issues in testsuite on Windows (when the index file could not be truncated). We should probably update the docs. Wrt performance - it *should* be slower than single file store because it does not have to keep keys in memory. How much highly depends on your access pattern (if you access the same subset of keys frequently it should not differ much).