Migrating FileCacheStore from 5.x to 7.2.x

[mjhaller]

I am trying to get a working file-based passivation Infinispan configuration for Hibernate Search 5.3/Infinispan 7.2.

Previously with Hibernate Search 4.1/Infinispan5 .x I had this config, which worked:

<?xml version="1.0" encoding="UTF-8"?>
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="urn:infinispan:config:5.1"
            xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd">

    <!-- *************************** -->
    <!-- System-wide global settings -->
    <!-- *************************** -->

    <global>
    
        <!-- Duplicate domains are allowed so that multiple deployments with default configuration 
            of Hibernate Search applications work - if possible it would be better to use JNDI to share 
            the CacheManager across applications -->
        <globalJmxStatistics
            enabled="false"
            cacheManagerName="HibernateSearch"
            allowDuplicateDomains="true" />

        <!-- If the transport is omitted, there is no way to create distributed or clustered 
            caches. There is no added cost to defining a transport but not creating a cache that uses one, 
            since the transport is created and initialized lazily. -->
        <transport 
        transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" 
            clusterName="Hibernate_Search_Infinispan_Cluster"
            distributedSyncTimeout="120000">
  <properties>
                <property name="configurationFile" value="jgroups-infinispan.xml" />
            </properties>
            <!-- Note that the JGroups transport uses sensible defaults if no configuration 
                property is defined. See the JGroupsTransport javadocs for more flags -->
        </transport>

        <!-- Used to register JVM shutdown hooks. hookBehavior: DEFAULT, REGISTER, DONT_REGISTER. 
            Hibernate Search takes care to stop the CacheManager so registering is not needed -->
        <shutdown
            hookBehavior="DONT_REGISTER" />

    </global>

    <!-- *************************** -->
    <!-- Default "template" settings -->
    <!-- *************************** -->

    <default>

        <loaders passivation="true" shared="false" preload="true">
          <loader class="org.infinispan.loaders.file.FileCacheStore"
              fetchPersistentState="true" ignoreModifications="false"
              purgerThreads="3" purgeSynchronously="true" purgeOnStartup="false">
              <properties>
                <property name="location" value="${hibernate.search.default.indexBase}" />
                <property name="fsyncMode" value="${hibernate.search.default.fsyncMode}" />
              </properties>
<!--             <singletonStore enabled="true" pushStateWhenCoordinator="true" pushStateTimeout="20000"/> -->
          </loader>
        </loaders>


        <locking
            lockAcquisitionTimeout="120000"
            writeSkewCheck="false"
            concurrencyLevel="5000"
            useLockStriping="false" />
            

        <!-- Invocation batching is required for use with the Lucene Directory -->
        <invocationBatching
            enabled="true" />

        <!-- This element specifies that the cache is clustered. modes supported: distribution 
            (d), replication (r) or invalidation (i). Don't use invalidation to store Lucene indexes (as 
            with Hibernate Search DirectoryProvider). Replication is recommended for best performance of 
            Lucene indexes, but make sure you have enough memory to store the index in your heap.
            Also distribution scales much better than replication on high number of nodes in the cluster. -->
        <clustering
            mode="replication">

            <!-- Prefer loading all data at startup than later -->
            <stateRetrieval
                timeout="120000"
                fetchInMemoryState="true"
                />

            <!-- Network calls are synchronous by default -->
            <sync
                replTimeout="20000" />
        </clustering>

        <jmxStatistics
            enabled="false" />

        <eviction
            maxEntries="-1"
            strategy="NONE" />

        <expiration
            maxIdle="-1" />

    </default>

    <!-- ******************************************************************************* -->
    <!-- Individually configured "named" caches.                                         -->
    <!--                                                                                 -->
    <!-- While default configuration happens to be fine with similar settings across the -->
    <!-- three caches, they should generally be different in a production environment.   -->
    <!--                                                                                 -->
    <!-- Current settings could easily lead to OutOfMemory exception as a CacheStore     -->
    <!-- should be enabled, and maybe distribution is desired.                           -->
    <!-- ******************************************************************************* -->

    <!-- *************************************** -->
    <!--  Cache to store Lucene's file metadata  -->
    <!-- *************************************** -->
    <namedCache
        name="LuceneIndexesMetadata">
        <clustering
            mode="replication">
            <stateRetrieval
                fetchInMemoryState="true" />
            <sync
                replTimeout="120000" />
        </clustering>
    </namedCache>

    <!-- **************************** -->
    <!--  Cache to store Lucene data  -->
    <!-- **************************** -->
    <namedCache
        name="LuceneIndexesData">
        <clustering
            mode="replication">
            <stateRetrieval
                fetchInMemoryState="true"/>
            <sync
                replTimeout="120000" />
        </clustering>
    </namedCache>

    <!-- ***************************** -->
    <!--  Cache to store Lucene locks  -->
    <!-- ***************************** -->
    <namedCache
        name="LuceneIndexesLocking">
        <clustering
            mode="replication">
            <stateRetrieval
                fetchInMemoryState="true"/>
            <sync
                replTimeout="120000" />
        </clustering>
    </namedCache>

</infinispan>

With Hibernate Search 5.3/Infinispan 7.2, the best I could come up with was the persistence stanza in the below xml, which is basically a copy of the default-hibernatesearch-infinispan.xml from the infinispan-directory-provider.jar.   However, unlike with the old config above, I see no files persisted when I terminate the tomcat instance which is running all of this.  Any guidance would be appreciated.

<?xml version="1.0" encoding="UTF-8"?>
<!--
 ~ Hibernate Search, full-text search for your domain model
 ~
 ~ License: GNU Lesser General Public License (LGPL), version 2.1 or later
 ~ See the lgpl.txt file in the root directory or <http://www.gnu.org/licenses/lgpl-2.1.html>.
  -->
<infinispan
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="urn:infinispan:config:7.2 http://www.infinispan.org/schemas/infinispan-config-7.2.xsd"
    xmlns="urn:infinispan:config:7.2">

    <!-- *************************** -->
    <!-- System-wide global settings -->
    <!-- *************************** -->
    <jgroups>
        <!-- Note that the JGroups transport uses sensible defaults if no configuration
            property is defined. See the JGroupsTransport javadocs for more flags.
            jgroups-udp.xml is the default stack bundled in the Infinispan core jar: integration
            and tuning are tested by Infinispan. -->
      <stack-file name="jgroups-dex-tcp" path="jgroups-infinispan.xml" />
    </jgroups>
    
    <cache-container name="HibernateSearch" default-cache="default" statistics="false" shutdown-hook="DONT_REGISTER">

        <transport cluster="Hibernate_Search_Infinispan_Cluster"  stack="jgroups-tcp"/>

        <!-- Duplicate domains are allowed so that multiple deployments with default configuration
            of Hibernate Search applications work - if possible it would be better to use JNDI to share
            the CacheManager across applications -->
        <jmx duplicate-domains="true" />
        
        <local-cache name="persisted_repository">
  <persistence passivation="true">
    <!-- note that class is missing and is induced by the fileStore element name -->
    <file-store
            shared="false" preload="true"
            fetch-state="true"
            read-only="true"
            purge="false"
            path="${hibernate.search.default.indexBase}">
             
        <write-behind flush-lock-timeout="15000" thread-pool-size="5" />
    </file-store>
  </persistence>  
        </local-cache>        

         <!-- *************************************** -->
         <!--  Cache to store Lucene's file metadata  -->
         <!-- *************************************** -->
         <replicated-cache name="LuceneIndexesMetadata" mode="SYNC" remote-timeout="25000">
            <transaction mode="NONE"/>
            <state-transfer enabled="true" timeout="480000" await-initial-transfer="true" />
            <indexing index="NONE" />
            <locking striping="false" acquire-timeout="10000" concurrency-level="500" write-skew="false" />
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
         </replicated-cache>

         <!-- **************************** -->
         <!--  Cache to store Lucene data  -->
         <!-- **************************** -->
         <replicated-cache name="LuceneIndexesData" mode="SYNC" remote-timeout="25000">
            <transaction mode="NONE"/>
            <state-transfer enabled="true" timeout="480000" await-initial-transfer="true" />
            <indexing index="NONE" />
            <locking striping="false" acquire-timeout="10000" concurrency-level="500" write-skew="false" />
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
         </replicated-cache>

         <!-- ***************************** -->
         <!--  Cache to store Lucene locks  -->
         <!-- ***************************** -->
        <replicated-cache name="LuceneIndexesLocking" mode="SYNC" remote-timeout="25000">
            <transaction mode="NONE"/>
            <state-transfer enabled="true" timeout="480000" await-initial-transfer="true" />
            <indexing index="NONE" />
            <locking striping="false" acquire-timeout="10000" concurrency-level="500" write-skew="false" />
            <eviction max-entries="-1" strategy="NONE"/>
            <expiration max-idle="-1"/>
        </replicated-cache>
    </cache-container>

</infinispan>

Thank you!

[rvansa]

The new implementation uses Single File Store that uses only one file (as the name suggests) and keeps all keys in-memory. Is this file present?

I don't have experience with async setup, so in case that the file is missing completely, please comment out the write-behind configuration and check if the sync version works.

[mjhaller]

Thanks, that is a good point about the Single File Store.  I touched a file to ensure it was there.  It didn't seem to help.  I also commented out the write-behind. Not sure what you meant by "sync version", but I am still not seeing passivation happening.  

[rvansa]

async = write behind

sync = without write behind

You don't need to touch anything; the path should point to a directory, where a file named cache-name.dat will be created automatically. If it is not, the cache has not been started, and you should inspect and post your log here.