8 Replies Latest reply on Aug 9, 2016 4:12 PM by Krishna Kalesh Balakrishnan

    Error: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper

    Krishna Kalesh Balakrishnan Newbie

      Hi,

       

      I am using Infinispan 8.2.2 in client-server mode with a master [2 nodes] and slave [2 nodes] .  I use replicated cache with indexing configuration as below.

      Data in getting indexed as well. But when I restart the Infinispan server, it throws me some Unable to open Lucene IndexReader exception. Could you please help.

       

      Configuration

      <subsystem xmlns="urn:infinispan:server:core:8.2">

        <cache-container name="clustered2" default-cache="default" statistics="true">

        <transport lock-timeout="60000"/>

        <replicated-cache-configuration name="replicated" mode="SYNC" start="EAGER"/>

       

        <replicated-cache name="ticket-cache" mode="ASYNC" start="EAGER">

             <indexing index="LOCAL">

                  <property name="default.indexmanager">org.infinispan.query.indexmanager.InfinispanIndexManager</property>

                  <property name="default.directory_provider">infinispan</property>

                  <property name="default.metadata_cachename">LuceneIndexesMetadataOWR1</property>

                  <property name="default.data_cachename">LuceneIndexesDataOWR1</property>

                  <property name="default.locking_cachename">LuceneIndexesLockingOWR1</property> 

                  <property name="default.exclusive_index_use">true</property>

                  <property name="lucene_version">LUCENE_CURRENT</property>

                  <property name="default.chunk_size">128000</property>

                  <property name="default.sharding_strategy.nbr_of_shards">2</property>

             </indexing>

             <locking acquire-timeout="60000"/>

             <expiration lifespan="43200000"/>

        </replicated-cache>

        <distributed-cache mode="SYNC" name="LuceneIndexesMetadataOWR1" start="EAGER" remote-timeout="30000">

        </distributed-cache>

        <distributed-cache mode="SYNC" name="LuceneIndexesDataOWR1" start="EAGER" remote-timeout="30000">

        </distributed-cache>

        <distributed-cache mode="SYNC" name="LuceneIndexesLockingOWR1" start="EAGER" remote-timeout="30000">

        </distributed-cache>

       

      Exception

      [Server:server2] ESC[31mCaused by: org.hibernate.search.exception.SearchException: HSEARCH000103: Unable to initialize IndexManager named 'org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper.0'ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:260)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:513)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManagers(IndexManagerHolder.java:482)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:91)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.spi.SearchIntegratorBuilder.initDocumentBuilders(SearchIntegratorBuilder.java:358)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.spi.SearchIntegratorBuilder.buildNewSearchFactory(SearchIntegratorBuilder.java:199)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.spi.SearchIntegratorBuilder.buildSearchIntegrator(SearchIntegratorBuilder.java:117)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.query.impl.LifecycleManager.getSearchFactory(LifecycleManager.java:303)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.query.impl.LifecycleManager.cacheStarting(LifecycleManager.java:115)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.factories.ComponentRegistry.notifyCacheStarting(ComponentRegistry.java:247)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:236)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:849)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:635)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:585)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:451)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.manager.impl.AbstractDelegatingEmbeddedCacheManager.getCache(AbstractDelegatingEmbeddedCacheManager.java:133)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.server.infinispan.SecurityActions$5.run(SecurityActions.java:130)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.server.infinispan.SecurityActions$5.run(SecurityActions.java:127)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.security.Security.doPrivileged(Security.java:76)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:63)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:135)ESC[0m

      [Server:server2] ESC[31m  at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:86)ESC[0m

      [Server:server2] ESC[31m  at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)ESC[0m

      [Server:server2] ESC[31m  at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)ESC[0m

      [Server:server2] ESC[31m  ... 3 moreESC[0m

      [Server:server2] ESC[31mCaused by: org.hibernate.search.exception.SearchException: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper.0ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:113)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.initialize(SharingBufferReaderProvider.java:91)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.PropertiesParseHelper.createDirectoryBasedReaderProvider(PropertiesParseHelper.java:172)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.createIndexReader(DirectoryBasedIndexManager.java:229)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:109)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)ESC[0m

      [Server:server2] ESC[31m  ... 26 moreESC[0m

      [Server:server2] ESC[31mCaused by: java.io.FileNotFoundException: Error loading metadata for index file: M|_5rb2.si|org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper.0|-1ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.lucene.impl.DirectoryImplementor.openInput(DirectoryImplementor.java:138)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.lucene.impl.DirectoryLucene.openInput(DirectoryLucene.java:102)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:109)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.codecs.lucene50.Lucene50SegmentInfoFormat.read(Lucene50SegmentInfoFormat.java:82)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:362)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:53)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:50)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:50)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.readerFactory(SharingBufferReaderProvider.java:131)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider$PerDirectoryLatestReader.<init>(SharingBufferReaderProvider.java:206)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:108)ESC[0m

      [Server:server2] ESC[31m  ... 31 moreESC[0m

      [Server:server2] ESC[31mESC[0m

      [Server:server2] ESC[31mESC[0mESC[0m15:59:30,148

       

       

      If I restart the server again : Another Exception

      [Server:server2] ESC[31mCaused by: org.hibernate.search.exception.SearchException: HSEARCH000103: Unable to initialize IndexManager named 'org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper.0'ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:260)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:513)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManagers(IndexManagerHolder.java:482)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.buildEntityIndexBinding(IndexManagerHolder.java:91)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.spi.SearchIntegratorBuilder.initDocumentBuilders(SearchIntegratorBuilder.java:358)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.spi.SearchIntegratorBuilder.buildNewSearchFactory(SearchIntegratorBuilder.java:199)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.spi.SearchIntegratorBuilder.buildSearchIntegrator(SearchIntegratorBuilder.java:117)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.query.impl.LifecycleManager.getSearchFactory(LifecycleManager.java:303)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.query.impl.LifecycleManager.cacheStarting(LifecycleManager.java:115)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.factories.ComponentRegistry.notifyCacheStarting(ComponentRegistry.java:247)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:236)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:849)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:635)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:585)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:451)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.manager.impl.AbstractDelegatingEmbeddedCacheManager.getCache(AbstractDelegatingEmbeddedCacheManager.java:133)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.server.infinispan.SecurityActions$5.run(SecurityActions.java:130)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.server.infinispan.SecurityActions$5.run(SecurityActions.java:127)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.security.Security.doPrivileged(Security.java:76)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:63)ESC[0m

      [Server:server2] ESC[31m  at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:135)ESC[0m

      [Server:server2] ESC[31m  at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:86)ESC[0m

      [Server:server2] ESC[31m  at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)ESC[0m

      [Server:server2] ESC[31m  at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)ESC[0m

      [Server:server2] ESC[31m  ... 3 moreESC[0m

      [Server:server2] ESC[31mCaused by: org.hibernate.search.exception.SearchException: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper.0ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:113)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.initialize(SharingBufferReaderProvider.java:91)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.PropertiesParseHelper.createDirectoryBasedReaderProvider(PropertiesParseHelper.java:172)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.createIndexReader(DirectoryBasedIndexManager.java:229)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.initialize(DirectoryBasedIndexManager.java:109)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.IndexManagerHolder.createIndexManager(IndexManagerHolder.java:256)ESC[0m

      [Server:server2] ESC[31m  ... 26 moreESC[0m

      [Server:server2] ESC[31mCaused by: java.io.IOException: Read past EOFESC[0m

      [Server:server2] ESC[31m  at org.infinispan.lucene.impl.SingleChunkIndexInput.readByte(SingleChunkIndexInput.java:54)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.store.DataInput.readInt(DataInput.java:101)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:194)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.codecs.CodecUtil.checkIndexHeader(CodecUtil.java:255)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.codecs.lucene50.Lucene50SegmentInfoFormat.read(Lucene50SegmentInfoFormat.java:86)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:362)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:53)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:50)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731)ESC[0m

       

       

      [Server:server2] ESC[31m  at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:50)ESC[0m

      [Server:server2] ESC[31m  at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.readerFactory(SharingBufferReaderProvider.java:131)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider$PerDirectoryLatestReader.<init>(SharingBufferReaderProvider.java:206)ESC[0m

      [Server:server2] ESC[31m  at org.hibernate.search.indexes.impl.SharingBufferReaderProvider.createReader(SharingBufferReaderProvider.java:108)ESC[0m

      [Server:server2] ESC[31m  ... 31 moreESC[0m

      [Server:server2] ESC[31m  Suppressed: org.apache.lucene.index.CorruptIndexException: checksum status indeterminate: remaining=0, please run checkindex for more details (resource=BufferedChecksumIndexInput(_4ktr.si))ESC[0m

      [Server:server2] ESC[31m          at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:370)ESC[0m

      [Server:server2] ESC[31m          at org.apache.lucene.codecs.lucene50.Lucene50SegmentInfoFormat.read(Lucene50SegmentInfoFormat.java:117)ESC[0m

      [Server:server2] ESC[31m          ... 40 moreESC[0m

      [Server:server2] ESC[31mESC[0m

        • 1. Re: Error: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
          Gustavo Fernandes Apprentice

          Hi, You are storing your data in a replicated cache, but the index is being stored on distributed caches. For replicated caches, you'd better store indexes on the filesystem rather than Infinispan itself and use a different IndexManager (the default):

           

          <indexing index="ALL">

                      <property name="default.indexmanager">near-real-time</property>

                      <property name="lucene_version">LUCENE_CURRENT</property>

                      <property name="default.chunk_size">128000</property>

          </indexing>

           

          With that config, each node will keep the index locally on disk (the index will be memory mapped by the OS), and will use a near real time reader for optimum performance.

           

          Could you try with this configuration?

          • 2. Re: Error: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
            Krishna Kalesh Balakrishnan Newbie

            Thanks Gustavo.

             

            But I tried the default settings as you specified, as my first option, but that was throwing another Exception.

            I use a .proto file to map to my object and index some of the fields in the object as well. Whenever I writes to cache for each row write, below exception occurs.

             

            That was the reason I tried - <property name="default.directory_provider">infinispan</property>

             

            Configuration

            <replicated-cache name="tickets-cache" mode="ASYNC" start="EAGER">

               <indexing index="ALL">

            <property name="default.indexmanager">near-real-time</property
            <property name="lucene_version">LUCENE_CURRENT</property>
            <property name="default.chunk_size">128000</property>
               </indexing>

               <locking acquire-timeout="60000"/>

               <expiration lifespan="43200000"/>

            </replicated-cache>

             

             

             

            Exception

            2016-08-04 20:59:08,318 ERROR [org.hibernate.search.exception.impl.LogErrorHandler] (Hibernate Search sync consumer thread for index org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper) HSEARCH000058: Exception occurred org.apache.lucene.store.LockObtainFailedException: Lock held by another program: D:\WorkSpace\Softwares\infinispan-server-8.2.2.Final\org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper\write.lock

            Primary Failure:

              Entity org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper  Id A:SgRCQUcx  Work Type  org.hibernate.search.backend.UpdateLuceneWork

            : org.apache.lucene.store.LockObtainFailedException: Lock held by another program: D:\WorkSpace\Softwares\infinispan-server-8.2.2.Final\org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper\write.lock

              at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:118)

              at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)

              at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)

              at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:776)

              at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:123)

              at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:89)

              at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)

              at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriterDelegate(AbstractWorkspaceImpl.java:203)

              at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:80)

              at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:46)

              at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.applyChangesets(SyncWorkProcessor.java:162)

              at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.run(SyncWorkProcessor.java:148)

              at java.lang.Thread.run(Thread.java:745)

             

             

            2016-08-04 20:59:08,320 ERROR [org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask] (Hibernate Search sync consumer thread for index org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper) HSEARCH000072: Couldn't open the IndexWriter because of previous error: operation skipped, index ouf of sync!

            2016-08-04 20:59:55,255 INFO  [org.jboss.as.clustering.infinispan] (remote-thread--p2-t13) DGISPN0001: Started ___event_log_cache cache from clustered container

            2016-08-04 20:59:55,403 INFO  [org.jboss.as.clustering.infinispan] (remote-thread--p2-t13) DGISPN0001: Started ___query_cache cache from clustered container

            2016-08-04 21:00:12,036 ERROR [org.hibernate.search.exception.impl.LogErrorHandler] (Hibernate Search sync consumer thread for index org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper) HSEARCH000058: Exception occurred org.apache.lucene.store.LockObtainFailedException: Lock held by another program: D:\WorkSpace\Softwares\infinispan-server-8.2.2.Final\org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper\write.lock

            Primary Failure:

              Entity org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper  Id A:SgRCQUcw  Work Type  org.hibernate.search.backend.UpdateLuceneWork

            : org.apache.lucene.store.LockObtainFailedException: Lock held by another program: D:\WorkSpace\Softwares\infinispan-server-8.2.2.Final\org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper\write.lock

              at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:118)

              at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)

              at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)

              at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:776)

              at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:123)

              at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:89)

              at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:117)

              at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriterDelegate(AbstractWorkspaceImpl.java:203)

              at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.applyUpdates(LuceneBackendQueueTask.java:80)

              at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueTask.run(LuceneBackendQueueTask.java:46)

              at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.applyChangesets(SyncWorkProcessor.java:162)

              at org.hibernate.search.backend.impl.lucene.SyncWorkProcessor$Consumer.run(SyncWorkProcessor.java:148)

              at java.lang.Thread.run(Thread.java:745)

            • 4. Re: Error: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
              Krishna Kalesh Balakrishnan Newbie

              Yes Gustavo. In local I use below setting in host.xml

               

              <servers>

                      <server name="server-one" group="cluster">        

                      </server>

                      <server name="server-two" group="cluster" auto-start="true">        

                          <socket-bindings port-offset="150"/>

                      </server>

                  </servers>

               

               

              Capture.PNG

               

              In higher env, below is the setting planned to use

              Capture.PNG

              • 5. Re: Error: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
                Gustavo Fernandes Apprentice

                In that case what is happening is that the two servers in the same node are using the same index directory and consequently stepping into each other's toes 

                 

                To solve that, try to configure the index location in each of the servers, for example:

                 

                On server1:

                <property name="default.indexBase">server1/index</property>

                 

                On server2:

                <property name="default.indexBase">server2/index</property>

                 

                and so on. You may be able to use a environment variable instead of hard coding the value, such as ${jboss.node.name} for example.

                • 6. Re: Error: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
                  Krishna Kalesh Balakrishnan Newbie

                  Thanks Gustavo for helping me out.  I have been trying out different possibilities for last couple of days.

                  As you suggested, I tried that option, but I am facing a stopper

                   

                  I have a single host.xml  with below nodes which use a single domain.xml. So ${jboss.node.name} is always the node name of

                  the coordinator node. In effect only folder gets created for storing the indexes and cause the lock issue again.

                   

                  host.xml - Attached

                  <servers>

                          <server name="server-one" group="cluster">  

                          </server>

                          <server name="server-two" group="cluster" auto-start="true">  

                              <socket-bindings port-offset="150"/>

                          </server>

                    </servers>

                  domain.xml -  Attached - Both the nodes use same domain.xml


                  Replication Mode

                  <replicated-cache name="tickets-cache" mode="ASYNC" start="EAGER">

                      <indexing index="ALL">

                          <property name="default.indexmanager">near-real-time</property>

                          <property name="lucene_version">LUCENE_CURRENT</property>

                          <property name="default.chunk_size">128000</property>

                          <property name="default.indexBase">${jboss.node.name}/index</property>

                      </indexing>

                      <locking acquire-timeout="60000"/>

                      <expiration lifespan="43200000"/>
                  </replicated-cache>

                   

                  So i tried with distributed cache .

                  <distributed-cache name="tickets-cache" mode="SYNC" start="EAGER">

                        <indexing index="LOCAL">

                              <property name="default.directory_provider">infinispan</property>

                              <property name="default.indexmanager">org.infinispan.query.indexmanager.InfinispanIndexManager</property>

                              <property name="lucene_version">LUCENE_CURRENT</property>

                              <property name="default.chunk_size">128000</property>

                        </indexing>

                        <locking acquire-timeout="60000"/>

                        <expiration lifespan="43200000"/>

                        <state-transfer timeout="900000" await-initial-transfer="true" />

                  </distributed-cache>

                   

                  Issue Faced

                  When tried with distributed cache, similar issue happens. Index is getting stored under only one node - the coordinator node.

                  When I read from cache using hotrod client, 1 out of 2 attempt fails as the coordinator node is not ready to share index with the other node.

                  • 7. Re: Error: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
                    Gustavo Fernandes Apprentice

                    Hello, there's no need to try with distributed cache.

                     

                    I tried the config files you provided (for the replicated cache), and after (re)starting the server (using bin/domain.sh), I could see two folders under the server root folder:

                    $ dir

                    bin  client  copyright.txt  docs  domain  jboss-modules.jar  LICENSE.txt  logs  master:server-one  master:server-two  modules  README.txt  rest  standalone  state

                     

                    each of "master:server-one" and "master:server-two" contains the index for a particular server:

                    $ dir master:server-one/index/org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper

                    segments_1  write.lock

                    $ dir master:server-two/index/org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper

                    segments_1  write.lock

                     

                    And it works normally.

                     

                    Can you see the same folders? Could you provide the exact steps to reproduce the issue?

                    • 8. Re: Error: Unable to open Lucene IndexReader for IndexManager org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
                      Krishna Kalesh Balakrishnan Newbie

                      Big  Thanks Gustavo. It is working good now.

                       

                      I was trying out in a windows machine where ":" is invalid character for folder name.

                      So I was trying to start each node with - domain.bat  -Djboss.node.name=server-one and domain.bat  -Djboss.node.name=server-two one after another,

                      so that  ${jboss.node.name} - will have value - server-one  - not master:server-one, which have caused some other issues.


                      Now I started using ${jboss.server.name}, whose value is server-one, server-two


                      <replicated-cache name="tickets-cache" mode="ASYNC" start="EAGER">

                          <indexing index="ALL">

                            <property name="default.indexmanager">near-real-time</property>

                            <property name="lucene_version">LUCENE_CURRENT</property>

                            <property name="default.chunk_size">128000</property>

                            <property name="default.indexBase">${jboss.server.name}/index</property>

                         </indexing>

                         <locking acquire-timeout="60000"/>

                         <expiration lifespan="43200000"/>

                      </replicated-cache>