Time Difference between Adding elements to Cache

[mvpratap]

I am having a cluster with 3 nodes In single system ... Adding one element of size 2 kb to cache takes 20-200 ms. Even 20 ms is not good for my app.This gave me bad performance impact..

I thought adding element to cache stored on local node(owner) may take less time compared with other node ...is this correct??

Is there any configuration to solve this. kindly provide some help.

[rvansa]

This is really low speed. Could you share your configuration? Maybe if you have indexing enabled, those inserts can be that slow.

[mvpratap]

infinispan-distribution.xml

=====================

<infinispan

        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

        xsi:schemaLocation="urn:infinispan:config:6.0 http://www.infinispan.org/schemas/infinispan-config-6.0.xsd"

        xmlns="urn:infinispan:config:6.0">

   <global>

   <transport clusterName="Mycluster">

   <properties>

   <property name="configurationFile" value="jgroups.xml"/>

    </properties>

     </transport>

   </global>

   <default>

        <locking useLockStriping="false"/>

     <clustering mode="distribution">

         <hash numOwners="1" numSegments="30"/>

        <sync/>

      </clustering>

   </default>

<namedCache name="myCache"/>

</infinispan>

jgroups.xml

========

<config xmlns="urn:org:jgroups"

        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.4.xsd">

    <TCP bind_addr="1.2.3.4"

         bind_port="7800" port_range="10"

         recv_buf_size="20000000"

         send_buf_size="640000"

         loopback="false"

         max_bundle_size="64k"

         bundler_type="old"

         enable_diagnostics="true"

         thread_naming_pattern="cl"

         timer_type="new"

         timer.min_threads="4"

         timer.max_threads="10"

         timer.keep_alive_time="3000"

         timer.queue_max_size="1000"

         timer.wheel_size="200"

         timer.tick_time="50"

         thread_pool.enabled="true"

         thread_pool.min_threads="2"

         thread_pool.max_threads="8"

         thread_pool.keep_alive_time="5000"

         thread_pool.queue_enabled="true"

         thread_pool.queue_max_size="100000"

         thread_pool.rejection_policy="discard"

oob_thread_pool.enabled="true"

         oob_thread_pool.min_threads="2"

         oob_thread_pool.max_threads="8"

         oob_thread_pool.keep_alive_time="5000"

         oob_thread_pool.queue_enabled="false"

         oob_thread_pool.queue_max_size="100"

         oob_thread_pool.rejection_policy="discard"/>

<!--   <MPING bind_addr="${jgroups.bind_addr:1.2.3.4}" break_on_coord_rsp="true"

          mcast_addr="${jgroups.mping.mcast_addr:228.2.4.6}"

          mcast_port="${jgroups.mping.mcast_port:43366}"

          ip_ttl="${jgroups.udp.ip_ttl:2}"

          num_initial_members="3" timeout="2000"/> -->

        <TCPPING initial_hosts="1.2.3.4[7800],1.2.3.5[7800]" port_range="2"

         timeout="3000" num_initial_members="3" />

    <MERGE2 max_interval="30000"

            min_interval="10000"/>

    <FD_SOCK/>

    <FD_ALL interval="4000" timeout="25000" />

    <VERIFY_SUSPECT timeout="500"  />

    <BARRIER />

    <pbcast.NAKACK use_mcast_xmit="false"

                   retransmit_timeout="100,300,600,1200"

                   discard_delivered_msgs="true" />

    <UNICAST3 conn_expiry_timeout="0"/>

    <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"

                   max_bytes="10m"/>

    <pbcast.GMS print_local_addr="true" join_timeout="5000"

                max_bundling_time="30"

                view_bundling="true"/>

    <UFC max_credits="2M"

         min_threshold="0.4"/>

    <MFC max_credits="2M"

         min_threshold="0.4"/>

<FRAG2 frag_size="60000"  />

    <pbcast.STATE_TRANSFER/>

</config>

[rvansa]

Try to remove TCP.bundler_type="old" (bundler names are mess across JGroups versions), and set TCP.oob_thread_pool.max_threads="50" just to check the bottleneck is not here.

[mvpratap]

Still facing same problem...It is taking more time (500 ms) when first time adding to cache..After that (20-80ms) per each entry.  On load test it is giving some what better result(10-30ms) how??.

Can any one provide some best configuration(jgroups and infinispan-distribution of cluster size 10) for  requirement .. I am having 40 caches each with storage of 1,00,000 elements...with 8,00,000 per minute put operatoins and 10,00,000 get operations per minute..