-
1. Re: near-real-time settings for Indexing with Infinispan Cache as backend
prashant.thakur Aug 18, 2015 6:38 AM (in response to prashant.thakur)Following is exception seen
INFO [org.infinispan.query.impl.massindex.IndexUpdater] [pool-16-thread-1] [2015-08-18 10:36:13,860] (IndexUpdater.java:48) - ISPN014016: Purging instances of 'org.infinispan.query.remote.indexing.ProtobufValueWrapper' from the index
ERROR [org.hibernate.search.exception.impl.LogErrorHandler] [pool-16-thread-1] [2015-08-18 10:36:18,997] (LogErrorHandler.java:67) - HSEARCH000058: HSEARCH000117: IOException on the IndexWriter
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@a357128
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:138)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:98)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:117)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:108)
at org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer.doWork(LuceneBackendTaskStreamer.java:45)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueProcessor.applyStreamWork(LuceneBackendQueueProcessor.java:76)
at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.performStreamOperation(DirectoryBasedIndexManager.java:107)
at org.infinispan.query.impl.massindex.ExtendedBatchBackend.purge(ExtendedBatchBackend.java:38)
at org.infinispan.query.impl.massindex.IndexUpdater.purge(IndexUpdater.java:49)
at org.infinispan.query.impl.massindex.IndexWorker.preIndex(IndexWorker.java:54)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:75)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:31)
at org.infinispan.commands.read.DistributedExecuteCommand.perform(DistributedExecuteCommand.java:98)
at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart$1.doLocalInvoke(DefaultExecutorService.java:1104)
at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart$1.call(DefaultExecutorService.java:1094)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
ERROR [org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer] [pool-16-thread-1] [2015-08-18 10:36:18,999] (LuceneBackendTaskStreamer.java:47) - HSEARCH000072: Couldn't open the IndexWriter because of previous error: operation skipped, index ouf of sync!
ERROR [org.hibernate.search.exception.impl.LogErrorHandler] [pool-16-thread-1] [2015-08-18 10:36:24,076] (LogErrorHandler.java:67) - HSEARCH000058: HSEARCH000117: IOException on the IndexWriter
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@a357128
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:138)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:98)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:117)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:108)
at org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer.doWork(LuceneBackendTaskStreamer.java:45)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueProcessor.applyStreamWork(LuceneBackendQueueProcessor.java:76)
at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.performStreamOperation(DirectoryBasedIndexManager.java:107)
at org.hibernate.search.backend.impl.batch.DefaultBatchBackend.flush(DefaultBatchBackend.java:76)
at org.infinispan.query.impl.massindex.ExtendedBatchBackend.flush(ExtendedBatchBackend.java:57)
at org.infinispan.query.impl.massindex.ExtendedBatchBackend.purge(ExtendedBatchBackend.java:42)
at org.infinispan.query.impl.massindex.IndexUpdater.purge(IndexUpdater.java:49)
at org.infinispan.query.impl.massindex.IndexWorker.preIndex(IndexWorker.java:54)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:75)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:31)
at org.infinispan.commands.read.DistributedExecuteCommand.perform(DistributedExecuteCommand.java:98)
at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart$1.doLocalInvoke(DefaultExecutorService.java:1104)
at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart$1.call(DefaultExecutorService.java:1094)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
-
2. Re: near-real-time settings for Indexing with Infinispan Cache as backend
gustavonalle Aug 18, 2015 1:09 PM (in response to prashant.thakur)It's technically possible to use the near-real-time index manager with Infinispan directory, but the caches that will hold the index (LuceneIndexesLocking,LuceneIndexesData and LuceneIndexesMetadata) need to be LOCAL, otherwise they will be competing to get the Lock when writing to the index.
-
3. Re: near-real-time settings for Indexing with Infinispan Cache as backend
prashant.thakur Aug 19, 2015 1:49 AM (in response to gustavonalle)the indexing starts but all entries are not getting indexed . We see locking issue after half of data is being indexed
Below is the error seen
ERROR [org.hibernate.search.exception.impl.LogErrorHandler] [remote-thread--p2-t4] [2015-08-19 05:46:50,333] (LogErrorHandler.java:67) - HSEARCH000058: HSEARCH000117: IOException on the IndexWriter
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@1bc8d0bf
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:138)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:98)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:117)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:108)
at org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer.doWork(LuceneBackendTaskStreamer.java:45)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueProcessor.applyStreamWork(LuceneBackendQueueProcessor.java:76)
at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.performStreamOperation(DirectoryBasedIndexManager.java:107)
at org.infinispan.query.impl.massindex.ExtendedBatchBackend.purge(ExtendedBatchBackend.java:38)
at org.infinispan.query.impl.massindex.IndexUpdater.purge(IndexUpdater.java:49)
at org.infinispan.query.impl.massindex.IndexWorker.preIndex(IndexWorker.java:54)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:75)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:31)
at org.infinispan.commands.read.DistributedExecuteCommand.perform(DistributedExecuteCommand.java:98)
at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
ERROR [org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer] [remote-thread--p2-t4] [2015-08-19 05:46:50,334] (LuceneBackendTaskStreamer.java:47) - HSEARCH000072: Couldn't open the IndexWriter because of previous error: operation skipped, index ouf of sync!
WARN [org.jgroups.protocols.MPING] [MPING] [2015-08-19 05:46:54,624] (Discovery.java:294) - a8140cda-41ac-7943-b66a-157b88da98d7: discarding discovery request for cluster 'MyGalaxy1' from a8140cda-41ac-7943-b66a-157b88da98d7; our cluster name is 'PST_CLUSTER_TEST'. Please separate your clusters properly
ERROR [org.hibernate.search.exception.impl.LogErrorHandler] [pool-16-thread-1] [2015-08-19 05:46:55,417] (LogErrorHandler.java:67) - HSEARCH000058: HSEARCH000117: IOException on the IndexWriter
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@1bc8d0bf
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:138)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:98)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:117)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:108)
at org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer.doWork(LuceneBackendTaskStreamer.java:45)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueProcessor.applyStreamWork(LuceneBackendQueueProcessor.java:76)
at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.performStreamOperation(DirectoryBasedIndexManager.java:107)
at org.infinispan.query.impl.massindex.ExtendedBatchBackend.purge(ExtendedBatchBackend.java:38)
at org.infinispan.query.impl.massindex.IndexUpdater.purge(IndexUpdater.java:49)
at org.infinispan.query.impl.massindex.IndexWorker.preIndex(IndexWorker.java:54)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:75)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:31)
at org.infinispan.commands.read.DistributedExecuteCommand.perform(DistributedExecuteCommand.java:98)
at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart$1.doLocalInvoke(DefaultExecutorService.java:1104)
at org.infinispan.distexec.DefaultExecutorService$LocalDistributedTaskPart$1.call(DefaultExecutorService.java:1094)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
ERROR [org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer] [pool-16-thread-1] [2015-08-19 05:46:55,418] (LuceneBackendTaskStreamer.java:47) - HSEARCH000072: Couldn't open the IndexWriter because of previous error: operation skipped, index ouf of sync!
ERROR [org.hibernate.search.exception.impl.LogErrorHandler] [remote-thread--p2-t1] [2015-08-19 05:47:00,511] (LogErrorHandler.java:67) - HSEARCH000058: HSEARCH000117: IOException on the IndexWriter
org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: org.infinispan.lucene.locking.BaseLuceneLock@1bc8d0bf
at org.apache.lucene.store.Lock.obtain(Lock.java:89)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:755)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.createNewIndexWriter(IndexWriterHolder.java:138)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:98)
at org.hibernate.search.backend.impl.lucene.IndexWriterHolder.getIndexWriter(IndexWriterHolder.java:117)
at org.hibernate.search.backend.impl.lucene.AbstractWorkspaceImpl.getIndexWriter(AbstractWorkspaceImpl.java:108)
at org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer.doWork(LuceneBackendTaskStreamer.java:45)
at org.hibernate.search.backend.impl.lucene.LuceneBackendQueueProcessor.applyStreamWork(LuceneBackendQueueProcessor.java:76)
at org.hibernate.search.indexes.spi.DirectoryBasedIndexManager.performStreamOperation(DirectoryBasedIndexManager.java:107)
at org.hibernate.search.backend.impl.batch.DefaultBatchBackend.flush(DefaultBatchBackend.java:76)
at org.infinispan.query.impl.massindex.ExtendedBatchBackend.flush(ExtendedBatchBackend.java:57)
at org.infinispan.query.impl.massindex.ExtendedBatchBackend.purge(ExtendedBatchBackend.java:42)
at org.infinispan.query.impl.massindex.IndexUpdater.purge(IndexUpdater.java:49)
at org.infinispan.query.impl.massindex.IndexWorker.preIndex(IndexWorker.java:54)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:75)
at org.infinispan.query.impl.massindex.IndexWorker.call(IndexWorker.java:31)
at org.infinispan.commands.read.DistributedExecuteCommand.perform(DistributedExecuteCommand.java:98)
at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
ERROR [org.hibernate.search.backend.impl.lucene.LuceneBackendTaskStreamer] [remote-thread--p2-t1] [2015-08-19 05:47:00,512] (LuceneBackendTaskStreamer.java:47) - HSEARCH000072: Couldn't open the IndexWriter because of previous error: operation skipped, index ouf of sync!
INFO [org.infinispan.query.impl.massindex.IndexUpdater] [remote-thread--p2-t1] [2015-08-19 05:47:00,588] (IndexUpdater.java:43) - ISPN014017: Flushing index 'org.infinispan.query.remote.indexing.ProtobufValueWrapper'
-
4. Re: near-real-time settings for Indexing with Infinispan Cache as backend
gustavonalle Aug 19, 2015 2:42 AM (in response to prashant.thakur)Hi, can you post your configuration?
-
5. Re: near-real-time settings for Indexing with Infinispan Cache as backend
prashant.thakur Aug 19, 2015 11:48 PM (in response to gustavonalle)the configurations from xml
<distributed-cache name="SUBSCRIBER" mode="SYNC" owners="2" segments="100" capacity="1" l1-lifespan="0"> <custom-interceptors> <interceptor after="org.infinispan.interceptors.compat.TypeConverterInterceptor" class="com.subex.spark.common.distributedcaching.server.interceptors.LastModifiedDateInterceptor"/> </custom-interceptors> <transaction transaction-manager-lookup="org.infinispan.transaction.lookup.GenericTransactionManagerLookup" mode="NON_XA" locking="OPTIMISTIC"/> </distributed-cache> <!-- *************************************** --> <!-- Cache to store Lucene's file metadata --> <!-- *************************************** --> <local-cache name="LuceneIndexesMetadata"> </local-cache> <!-- **************************** --> <!-- Cache to store Lucene data --> <!-- **************************** --> <local-cache name="LuceneIndexesData"> </local-cache> <local-cache name="LuceneIndexesLocking"> </local-cache> From Java code related to index configuration..
SearchMapping mapping = new SearchMapping();
mapping.entity(classType).indexed().providedId() ;
Properties props = new Properties();
props.put(org.hibernate.search.cfg.Environment.MODEL_MAPPING, mapping);
props.put("hibernate.search.default.directory_provider", "infinispan");
props.put("hibernate.search.default.indexmanager", "near-real-time");
configurationBuilder.indexing().index(Index.LOCAL).autoConfig(true).withProperties(props);