0 Replies Latest reply on Dec 11, 2013 1:54 AM by guzhang

    How to optimize building index for object query in client-server mode in infinispan 6.0 final?

    guzhang

      My requirements:

      1).put the Person object in cache

         fields:

         String name;

         String cert;

         String phone;

         String address;

      2).query Person object by these fields

      3).Person records:5,000,000

      ===================================

      So,I use remote querying function.

      config:

      <subsystem xmlns="urn:infinispan:server:core:5.3" default-cache-container="local">
      <cache-container name="local" default-cache="default">
      <local-cache name="default" start="EAGER">
      <locking isolation="NONE" acquire-timeout="30000" concurrency-level="1000" striping="false"/>
      <transaction mode="NONE"/>
      <indexing index="LOCAL">
      <property name="hibernate.search.jmx_enabled">true</property>
      <property name="hibernate.search.default.directory_provider">ram</property>
      <property name="hibernate.search.lucene_version">LUCENE_CURRENT</property><!-- LUCENE_30 -->
      <property name="hibernate.search.default.exclusive_index_use">true</property>
      <property name="hibernate.search.default.indexwriter.use_compound_file">false</property>
      <property name="hibernate.search.default.indexwriter.ram_buffer_size">256</property>
      </indexing>
      </local-cache>

      -----------------------------

      Java Code:

      ...
      remoteCache.put(id,person);
      ...
      QueryFactory queryFactory = Search.getQueryFactory(remoteCache);
      Query query = queryFactory.from (Person.class)
      .having("name").eq("john")
      .and()
      .having("cert").eq("4349823211")
      .toBuilder().build();

       

      ===================================

      Then,I put the Person data in cache,but the performance is too low,I can not load these data in 30 minutes.

      load:100,000    200,000    300,000    500,000   ...5,000,000

      cost:3min         9min          19min       47            >10hours

      --------------

      If index="NONE" I can put all the data in 6 minutes.How can I optimize building index?

      or can I put the data by the following way?

      1)disable building index

      2)put the data

      3)enable building index

      4)build index

       

      or ...?

       

      Any suggestion would be greatly appreciated.