0 Replies Latest reply on Mar 19, 2013 5:58 PM by Trevor Campbell

    Is Infinispan's Lucene directory suitable and performant in a high write environment

    Trevor Campbell Newbie

      We have been doing some testing with Infinisan's Lucene Directory in an attempt to scale our application further.  Currently it uses Lucene with a FSDirectory.


      We use Lucene with a heavy write load and require the index to be consistent so that updated documents appear immeditaley in any following search, so user transactions wait for a document to be added or updated before returning.  Many of our searchers also return a very high protion of documents in the index. Customers may have between 200,000 and 2,000,000 documents in the index.  Locally we get good performance < 0.5 secs for most operations.


      We also need our index to be persistent as the time for a total rebuild can be excessive.


      It seems from our testing that this does not suite Infinispan's Lucene directory because we need to user REPL_SYNC on the cache and that the continuous replication of updates between nodes just adds too much overhead, with this application characteristic.


      I was wondering if others though this to be a reasonable finding, or are we possibly missing something fundamental.