I am new to ISPN, and I am attempting to setup a large (20-40GB) distributed persistent cache with a non-shared local cache store. I'm using the ISPN that is bundled with EAP 6.1. I am trying to understand how to manage the operational aspects of the distributed cache - starting/stopping (no topology change) versus reconfiguring (topology change).
1) When starting the cluster, I want the individual ISPN nodes (each running in EAP) to load whatever data is stored in the local store, and not get their state from other nodes in the cluster. I just want them to start warm with the same state as when they were stopped. This seems to imply fetchInMemoryState = false, however this contradicts (3) below. This also suggests preload = true, and fetchPersistentState = false for the loader configuration.
2) When stopping the cluster, I want the individual ISPN nodes (each running in EAP) to stop without transferring their data other nodes. (It seems that if I don't somehow prevent the state transfer, the entire cache data (20-40GB) will be shuffled around until it finally all ends up at the last node to leave, with the opposite happening when the cluster is restarted.) This again seems to imply fetchInMemoryState = false, and fetchPersistentState = false for the loader configuration, but this contradicts (3) below.
3) When reconfiguring the cache (adding and removing nodes), I do want the data re-balanced. This implies fetchInMemoryState = true, and fetchPersistentState = true for the loader configuration, which contradicts the above.
I have read ISPN-1239, Controlled cluster shutdown with data restore from persistent storage, ISPN-3351, and infinispan5.3 fetchInMemoryState=false,some data loss. These seem to imply that (2) is possible by disabling re-balancing using JMX. (3) is obviously possible with state transfer enabled. But I do not see how (1) is possible, since the JMX operation applies to a running cache - EAP is going to start the node with state transfer enabled before I can disable it with JMX.
As a newbie, am I missing something obvious and does what I am trying to do make sense? Any suggestions how to accomplish the above would be greatly appreciated.
Also, as a side note, it's unclear to me the distinction between fetchPersistentState and fetchInMemoryState. In searching the source code it appears that these flags are equivalent, and if either (or both) are set, it results in org.infinispan.statetransfer.StateConsumerImpl.isFetchEnabled being set to true. Is there a distinction or have I missed something?