0 Replies Latest reply on Jun 24, 2013 3:08 PM by cluc

    Problems with distributed index initialization

    cluc
    cluc Jun 24, 2013 3:08 PM

      Hi all,

       

      I'm trying to configure 2 caches with indexing. I configured Infinispan as directory provider for the index with jgroups as worker backend. When i work with the "namedCache1" to initialize the index, first i run the "master node" and put some item in the cache then i run the "slave node" and in the startup the clusters (my_cluster, hibernate-search-cluster) are formed. Then i'm able to run the initial load (from jdbc to cluster), this procedure works fine with "namedCache1" but didnt work for "namedCache2". First, in the startup only my_cluster is formed and later when i put some item to "namedCache2" the MasterSwitchDelegatingQueueProcessor (applyWork()) sends the updates to the "remoteMaster" and then the index store didn't initialize correctly.

       

      Versions:

       

           JRE 1.6.0 IBM J9 2.4 Linux x86-32

           Jboss 4.3

           Infinispan 5.2.1.Final

           Infinsipan query 5.2.1.Final

           Hibernate Search 4.2.0.Final

       

       

      I set up the nameds caches and the caches for the index in the same index becouse of

       

      https://community.jboss.org/thread/220368

       

      This is the configuration for the master node:

       

      Master node

      <?xml version="1.0" encoding="UTF-8"?>

      <infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

          xsi:schemaLocation="urn:infinispan:config:5.2 http://www.infinispan.org/schemas/infinispan-config-5.2.xsd"

          xmlns="urn:infinispan:config:5.2">

       

          <global>

       

              <transport clusterName="my_cluster" nodeName="name-A"

                  transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport">

                  <properties>

                      <property name="configurationFile" value="jgroups-tcp.xml" />

                  </properties>

              </transport>

       

              <globalJmxStatistics enabled="true" jmxDomain="org.infinispan"

                  cacheManagerName="SampleCacheManager" mBeanServerLookup="org.infinispan.jmx.JBossMBeanServerLookup" />

       

          </global>

       

          <default>

       

              <storeAsBinary />

       

              <locking isolationLevel="READ_COMMITTED"

                  lockAcquisitionTimeout="200000" writeSkewCheck="false"

                  concurrencyLevel="5000" useLockStriping="false" />

       

              <transaction

                  transactionManagerLookupClass="org.infinispan.transaction.lookup.GenericTransactionManagerLookup"

                  syncRollbackPhase="false" syncCommitPhase="false" cacheStopTimeout="30000"

                  use1PcForAutoCommitTransactions="false" autoCommit="true"

                  lockingMode="OPTIMISTIC" useSynchronization="false" transactionMode="TRANSACTIONAL"

                  useEagerLocking="true" eagerLockSingleNode="true">

                  <recovery enabled="false" />

              </transaction>

       

              <jmxStatistics enabled="true" />

       

              <invocationBatching enabled="false" />

       

              <indexing enabled="true" indexLocalOnly="true">

       

                  <properties>

       

                      <property name="hibernate.search.default.indexmanager"

                          value="org.infinispan.query.indexmanager.InfinispanIndexManager" />

       

                      <!-- specify the managed index is to be shared across the nodes -->

                      <property name="hibernate.search.default.directory_provider"

                          value="infinispan" />

       

                      <!-- Supporting exclusive index usage will require lock cleanup on crashed

                          nodes to be implemented -->

                      <property name="hibernate.search.default.exclusive_index_use"

                          value="true" />

       

                      <property name="hibernate.search.infinispan.chunk_size"

                          value="4096" />

       

                      <property name="hibernate.search.lucene_version" value="LUCENE_CURRENT" />

       

                      <property name="hibernate.search.default.worker.backend"

                          value="jgroupsMaster" />

       

                      <property name="hibernate.search.worker.execution" value="async" />

       

                      <property name="hibernate.search.services.jgroups.configurationFile"

                          value="infinispan/jgroups-tcp-custom.xml" />

       

                      <property name="hibernate.search.services.jgroups.clusterName"

                          value="hibernate-search-cluster" />

       

                      <property name="hibernate.search.jmx_enabled" value="true" />

       

              </indexing>

       

              <eviction strategy="NONE" />

              <expiration maxIdle="-1" reaperEnabled="false" />

       

              <clustering mode="dist">

                  <async />

                  <hash numOwners="1" />

              </clustering>

       

          </default>

       

          <namedCache name="namedCache1">

              <loaders passivation="false" shared="false" preload="true">

                  <store class="org.infinispan.loaders.file.FileCacheStore"

                      fetchPersistentState="false" ignoreModifications="false"

                      purgeOnStartup="false">

                      <properties>

                          <property name="location" value="${jboss.server.data.dir}" />

                      </properties>

                      <async enabled="true" flushLockTimeout="15000" threadPoolSize="3" />

                  </store>

              </loaders>

          </namedCache>

       

          <namedCache name="namedCache2">

              <loaders passivation="false" shared="false" preload="true">

                  <store class="org.infinispan.loaders.file.FileCacheStore"

                      fetchPersistentState="false" ignoreModifications="false"

                      purgeOnStartup="false">

                      <properties>

                          <property name="location" value="${jboss.server.data.dir}" />

                      </properties>

                      <async enabled="true" flushLockTimeout="15000" threadPoolSize="3" />

                  </store>

              </loaders>

          </namedCache>

       

          <namedCache name="LuceneIndexesMetadata">

              <clustering mode="replication">

                  <stateTransfer fetchInMemoryState="true" />

                  <async />

              </clustering>

              <indexing enabled="false" />

              <eviction maxEntries="-1" strategy="NONE" />

              <expiration maxIdle="-1" reaperEnabled="false" />

              <jmxStatistics enabled="true" />

          </namedCache>

       

          <namedCache name="LuceneIndexesData">

              <clustering mode="dist">

                  <sync replTimeout="25000" />

                  <stateTransfer fetchInMemoryState="true" />

                  <hash numOwners="1" />

              </clustering>

              <indexing enabled="false" />

              <eviction maxEntries="-1" strategy="NONE" />

              <expiration maxIdle="-1" reaperEnabled="false" />

              <jmxStatistics enabled="true" />

          </namedCache>

       

          <namedCache name="LuceneIndexesLocking">

              <clustering mode="replication">

                  <stateTransfer fetchInMemoryState="true" />

                  <async />

              </clustering>

              <indexing enabled="false" />

              <eviction maxEntries="-1" strategy="NONE" />

              <expiration maxIdle="-1" reaperEnabled="false" />

              <jmxStatistics enabled="true" />

          </namedCache>

      </infinispan>

       

      And this is the configuration file for the slave node:

       

      Slave node

      <?xml version="1.0" encoding="UTF-8"?>

      <infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

          xsi:schemaLocation="urn:infinispan:config:5.2 http://www.infinispan.org/schemas/infinispan-config-5.2.xsd"

          xmlns="urn:infinispan:config:5.2">

       

          <global>

       

              <transport clusterName="my-cluster" nodeName="name-B"

                  transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport">

                  <properties>

                      <property name="configurationFile" value="jgroups-tcp.xml" />

                  </properties>

              </transport>

       

              <globalJmxStatistics enabled="true" jmxDomain="org.infinispan"

                  cacheManagerName="SampleCacheManager" mBeanServerLookup="org.infinispan.jmx.JBossMBeanServerLookup" />

       

          </global>

          <default>

       

              <storeAsBinary />

       

              <locking isolationLevel="READ_COMMITTED"

                  lockAcquisitionTimeout="200000" writeSkewCheck="false"

                  concurrencyLevel="5000" useLockStriping="false" />

       

              <transaction

                  transactionManagerLookupClass="org.infinispan.transaction.lookup.GenericTransactionManagerLookup"

                  syncRollbackPhase="false" syncCommitPhase="false" cacheStopTimeout="30000"

                  use1PcForAutoCommitTransactions="false" autoCommit="true"

                  lockingMode="OPTIMISTIC" useSynchronization="false" transactionMode="TRANSACTIONAL"

                  useEagerLocking="true" eagerLockSingleNode="true">

                  <recovery enabled="false" />

              </transaction>

       

              <jmxStatistics enabled="true" />

       

              <invocationBatching enabled="false" />

       

              <indexing enabled="true" indexLocalOnly="true">

                  <properties>

                      <property name="hibernate.search.default.indexmanager"

                          value="org.infinispan.query.indexmanager.InfinispanIndexManager" />

       

                      <property name="hibernate.search.default.directory_provider"

                          value="infinispan" />

       

                      <property name="hibernate.search.default.exclusive_index_use"

                          value="false" />

       

                      <property name="hibernate.search.infinispan.chunk_size"

                          value="4096" />

       

                      <property name="hibernate.search.lucene_version" value="LUCENE_CURRENT" />

       

                      <property name="hibernate.search.default.worker.backend"

                          value="jgroupsSlave" />

       

                      <property name="hibernate.search.worker.execution" value="async" />

       

                      <property name="hibernate.search.services.jgroups.configurationFile"

                          value="infinispan/jgroups-tcp-custom.xml" />

       

                      <property name="hibernate.search.services.jgroups.clusterName"

                          value="hibernate-search-cluster" />

       

                      <property name="hibernate.search.jmx_enabled" value="true" />

                  </properties>

              </indexing>

       

              <eviction strategy="NONE" />

              <expiration maxIdle="-1" reaperEnabled="false" />

       

              <clustering mode="dist">

                  <async />

                  <hash numOwners="1" />

              </clustering>

          </default>

       

          <namedCache name="namedCache1">

              <loaders passivation="false" shared="false" preload="true">

                  <store class="org.infinispan.loaders.file.FileCacheStore"

                      fetchPersistentState="false" ignoreModifications="false"

                      purgeOnStartup="false">

                      <properties>

                          <property name="location" value="${jboss.server.data.dir}" />

                      </properties>

                      <async enabled="true" flushLockTimeout="15000" threadPoolSize="3" />

                  </store>

              </loaders>

          </namedCache>

       

          <namedCache name="namedCache2">

              <loaders passivation="false" shared="false" preload="true">

                  <store class="org.infinispan.loaders.file.FileCacheStore"

                      fetchPersistentState="false" ignoreModifications="false"

                      purgeOnStartup="false">

                      <properties>

                          <property name="location" value="${jboss.server.data.dir}"/>

                      </properties>

                      <async enabled="true" flushLockTimeout="15000" threadPoolSize="3"/>

                  </store>

              </loaders>

          </namedCache>

       

          <namedCache name="LuceneIndexesMetadata">

              <clustering mode="replication">

                  <stateTransfer fetchInMemoryState="true" />

                  <async />

              </clustering>

       

              <indexing enabled="false" />

       

              <eviction maxEntries="-1" strategy="NONE" />

       

              <expiration maxIdle="-1" reaperEnabled="false" />

          </namedCache>

       

          <namedCache name="LuceneIndexesData">

              <clustering mode="dist">

                  <sync replTimeout="25000" />

                  <stateTransfer fetchInMemoryState="true" />

                  <hash numOwners="1" />

              </clustering>

       

              <indexing enabled="false" />

       

              <eviction maxEntries="-1" strategy="NONE" />

       

              <expiration maxIdle="-1" reaperEnabled="false" />

          </namedCache>

       

          <namedCache name="LuceneIndexesLocking">

              <clustering mode="replication">

                  <stateTransfer fetchInMemoryState="true" />

                  <async />

              </clustering>

       

              <indexing enabled="false" />

       

              <eviction maxEntries="-1" strategy="NONE" />

       

              <expiration maxIdle="-1" reaperEnabled="false" />

       

          </namedCache>

      </infinispan>

       

      What am i missing to obtain the correct initialization of the index on namedCache 2?

       

      Appreciate some help, thanks

      Charles.-