4 Replies Latest reply on Jan 31, 2011 12:59 PM by Galder Zamarreño

    Increasing performance in bulk data inserts

    havell Newbie

      We are testing infinispan platform in order to sustitute Coherence (from Oracle). One of our apps consumes data directly from Cache, and it requires about 14.000.000 rows. In production environment, we managed to make about 4000 inserts/s (it takes one hour). I have seen some infinispan benchmarks close to that value.

       

      We opted for 4 nodes topology with distributed policy (4 virtual machines running on ESXi servers, private LAN), L1 cache disabled and protocol hotrod:

       

      <namedCache name="test">

            <clustering mode="distribution">

               <sync/>

               <hash rehashEnabled="false" rehashRpcTimeout="60000" numOwners="2"/>

               <l1 enabled="false"/>

            </clustering>

        </namedCache>

       

      However, in our tests we only could achieve  400 inserts/s (for us, a poor value).

      Maybe we have a mistaken focus. We are trying to insert the rows from hotrod-client (we tested "put" and "putAll" methods). Maybe there is a way to load all data directly from file (or DB...) to Cache, and that way would be quicker.

       

      We reviewed all documentation, and we didn't find any clue, could you help us?

       

      Thanks