4 Replies Latest reply on Aug 7, 2014 9:06 AM by infinispan user

    Time Difference between Adding elements to Cache

    infinispan user Newbie

      I am having a cluster with 3 nodes In single system ... Adding one element of size 2 kb to cache takes 20-200 ms. Even 20 ms is not good for my app.This gave me bad performance impact..

       

      I thought adding element to cache stored on local node(owner) may take less time compared with other node ...is this correct??

       

      Is there any configuration to solve this. kindly provide some help.

        • 1. Re: Time Difference between Adding elements to Cache
          Radim Vansa Master

          This is really low speed. Could you share your configuration? Maybe if you have indexing enabled, those inserts can be that slow.

          • 2. Re: Time Difference between Adding elements to Cache
            infinispan user Newbie

            infinispan-distribution.xml

            =====================

            <infinispan

                    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

                    xsi:schemaLocation="urn:infinispan:config:6.0 http://www.infinispan.org/schemas/infinispan-config-6.0.xsd"

                    xmlns="urn:infinispan:config:6.0">

             

             

               <global>

               <transport clusterName="Mycluster">

               <properties>

               <property name="configurationFile" value="jgroups.xml"/>

                </properties>

                 </transport>

               </global>

             

             

               <default>

                    <locking useLockStriping="false"/>

                 <clustering mode="distribution">

                     <hash numOwners="1" numSegments="30"/>

                    <sync/>

                  </clustering>

               </default>

            <namedCache name="myCache"/>

            </infinispan>

             

            jgroups.xml

            ========

            <config xmlns="urn:org:jgroups"

                    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

                    xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/JGroups-3.4.xsd">

                <TCP bind_addr="1.2.3.4"

                     bind_port="7800" port_range="10"

                     recv_buf_size="20000000"

                     send_buf_size="640000"

                     loopback="false"

                     max_bundle_size="64k"

                     bundler_type="old"

                     enable_diagnostics="true"

                     thread_naming_pattern="cl"

             

             

                     timer_type="new"

                     timer.min_threads="4"

                     timer.max_threads="10"

                     timer.keep_alive_time="3000"

                     timer.queue_max_size="1000"

                     timer.wheel_size="200"

                     timer.tick_time="50"

             

             

                     thread_pool.enabled="true"

                     thread_pool.min_threads="2"

                     thread_pool.max_threads="8"

                     thread_pool.keep_alive_time="5000"

                     thread_pool.queue_enabled="true"

                     thread_pool.queue_max_size="100000"

                     thread_pool.rejection_policy="discard"

            oob_thread_pool.enabled="true"

                     oob_thread_pool.min_threads="2"

                     oob_thread_pool.max_threads="8"

                     oob_thread_pool.keep_alive_time="5000"

                     oob_thread_pool.queue_enabled="false"

                     oob_thread_pool.queue_max_size="100"

                     oob_thread_pool.rejection_policy="discard"/>

             

             

            <!--   <MPING bind_addr="${jgroups.bind_addr:1.2.3.4}" break_on_coord_rsp="true"

                      mcast_addr="${jgroups.mping.mcast_addr:228.2.4.6}"

                      mcast_port="${jgroups.mping.mcast_port:43366}"

                      ip_ttl="${jgroups.udp.ip_ttl:2}"

                      num_initial_members="3" timeout="2000"/> -->

             

             

                    <TCPPING initial_hosts="1.2.3.4[7800],1.2.3.5[7800]" port_range="2"

                     timeout="3000" num_initial_members="3" />

                <MERGE2 max_interval="30000"

                        min_interval="10000"/>

             

             

                <FD_SOCK/>

                <FD_ALL interval="4000" timeout="25000" />

                <VERIFY_SUSPECT timeout="500"  />

                <BARRIER />

                <pbcast.NAKACK use_mcast_xmit="false"

                               retransmit_timeout="100,300,600,1200"

                               discard_delivered_msgs="true" />

                <UNICAST3 conn_expiry_timeout="0"/>

             

             

                <pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"

                               max_bytes="10m"/>

                <pbcast.GMS print_local_addr="true" join_timeout="5000"

                            max_bundling_time="30"

                            view_bundling="true"/>

                <UFC max_credits="2M"

                     min_threshold="0.4"/>

                <MFC max_credits="2M"

                     min_threshold="0.4"/>

            <FRAG2 frag_size="60000"  />

                <pbcast.STATE_TRANSFER/>

            </config>

            • 3. Re: Time Difference between Adding elements to Cache
              Radim Vansa Master

              Try to remove TCP.bundler_type="old" (bundler names are mess across JGroups versions), and set TCP.oob_thread_pool.max_threads="50" just to check the bottleneck is not here.

              • 4. Re: Time Difference between Adding elements to Cache
                infinispan user Newbie

                Still facing same problem...It is taking more time (500 ms) when first time adding to cache..After that (20-80ms) per each entry.  On load test it is giving some what better result(10-30ms) how??.

                 

                Can any one provide some best configuration(jgroups and infinispan-distribution of cluster size 10) for  requirement .. I am having 40 caches each with storage of 1,00,000 elements...with 8,00,000 per minute put operatoins and 10,00,000 get operations per minute..