CacheLoader at cache startup

[ferwasy]

Hello. I am trying to understand and determine how the load of entries at a single cache node startup (regardless of the cluster mode is configured) works.

What I have found is the following:

-If the configuration option "preload" is set to false, then no entry will be loaded at startup time, and them will be loaded lazily "on-demand".

-If the configuration option "preload" is set to true, then the eviction configuration is taken into account. If eviction is disable, then all entries from the underlying store are loaded. If eviction is enable, then "maxEntries" (from eviction configuration) entries are loaded, without guarantee any particular order.

But, most important (for me) is that, as far as I understand, the cluster mode (replication o distribution) has no incidence in this functionality. So, if I am right, in a distributed cluster, a new cache node joining will load all entries from the underlying store, "blurring" in some degree the frontier between replication and distribution.

Am I right? Answers are more than welcome.

Kind regards.

Fernando.

PS: the references for my thoughts are the following:

• https://docs.jboss.org/author/display/ISPN/CacheLoaders
• https://community.jboss.org/thread/194896 
• Class org.infinispan.loaders.CacheLoaderManagerImpl

[mgencur]

Hi Fernando,

feel free to look at https://github.com/infinispan/infinispan/blob/master/core/src/test/java/org/infinispan/distribution/PessimisticDistSyncTxCacheStoreSharedTest.java and play with it. You'll find out the entries that were in the first node and its underlying cache store are redistributed after the new node joins the cluster. There should not be duplicated records when numOwners=1. The parameter called fetchPersistentState also plays its role here.

[ferwasy]

Martin: if the cache stored is shared among all nodes/instances, all nodes load all the entries at startup. I have tested this by starting 3 nodes in (aprox) the same time. No info about the replicated/distribution modes is taken into account in the process of preload entries (CacheLoadManagerImpl.preload()). I am using the 5.1.x version obtained from the branch in git.

Regards.

Fernando.

[mgencur]

Fernando,

this is strange. I modified the test above (https://github.com/mgencur/infinispan/commit/c8025e10982dde257e5919e0aecf58bb24e18959) and when I run it, I get output similar to:

Size1:1

Size2:3

Size1:2

Size2:6

Other times I get this:

Size1:2

Size2:2

Size1:4

Size2:4

or

Size1:3

Size2:1

Size1:4

Size2:4

It is a distributed mode so the distribution depends on a hash-wheel algorithm. Can you provide a test that proves your theses? e.g. similar to the above one

[ferwasy]

Martin: there is no need for the test. If you have a shared cache loader with preload=true, CacheLoaderManagerImpl preloads all entries. I think this should be different in case "distribution" mode is selected. Maybe preload entries based on the consistent hash? (only those which are owned by the node)?

[ferwasy]

For more info, this is the relevant parts of my configuration.

<clustering mode="distribution">

     <hash numOwners="1"/>

     <sync />

     <stateTransfer fetchInMemoryState="true"/>

</clustering>

<loaders passivation="false" shared="true" preload="true">

     <loader class="org.infinispan.loaders.jdbc.mixed.JdbcMixedCacheStore"

     fetchPersistentState="false" ignoreModifications="false"

     purgeOnStartup="false">

Using Infinispan 5.1.5