1 2 Previous Next 16 Replies Latest reply on Aug 23, 2007 6:10 PM by Lior Neumann

    Performance expectations

    Lior Neumann Newbie

      Hi,

      We have been evaluating JBC for production use in our system, and have decided to test PojoCache performance on the hardware we intend to use.

      Looking for performance data, we came across this performance test:
      http://wiki.jboss.org/wiki/Wiki.jsp?page=WhatShouldWeExpectOfThePojoCachePerformance

      The reason for this post is that we tried to run our own, differnt, throughput test, but the performance seemed very degraded in comparison to the expected results, implied by the above reference.
      The two tests are not identical, but based on the reference we believe our test should have run significantly faster than it does.
      In practice, the test runs about four time slower than it should take to attach these amount of new, very small POJOs.

      Can anyone please point out what we have been doing wrong, verify our results, cite other benchmarks of JBoss-Cache etc. ?
      Also, if anyone knows about comparison to other caching solutions, we would be interested.


      Below are the test details, our results, the configuration file and the test code itself.


      Thank you in advance,

      Lior Neuman
      R&D Team
      MailVision LTD


      Test environment:
      Single node (Local cache),
      HW: Dual P4 Xeon 3.0, 2Gb, 80Gb
      SW: running Linux 2.6.9-22.ELsmp, JVM: 32bit 1.5.0_06

      Remarks:
      Our HW has 2G RAM whereas the test had 4G, but only 20% of it are being used throughout the test.
      The system was dedicated for the test.
      Logging level is set to INFO, sent to console only

      Test Plan and Load Pattern
      A Local cache, with optimistic-locking and no eviction is created and started.
      A single thread, with no added delay, invokes putObject(key, value) where both key and value are new String instances, which have not been attached before.

      Basically, the code that runs is the following, in an infinite loop: (full code at the end)

      String key = "/root/Key-" + counter;
      Object value = "dummy" + counter;
      
      pojoCache.putObject(key, value);
      
      if (counter % 1000 == 0) {
       // print counter and elapsed time
      }
      



      Test Results

      Col1 = Total Objs
      Col2 = Time Delta[sec]
      Col3 = Throughput [obj/sec]

      Col 1 | Col 2 | Col 3
      ------------------------------
      1000 | 4.84 | 206.75
      2000 | 2.38 | 277.2
      3000 | 2.02 | 324.89
      4000 | 1.95 | 357.55
      5000 | 2.28 | 371.25
      6000 | 2.68 | 371.56
      7000 | 3.14 | 363
      8000 | 3.52 | 350.82
      9000 | 3.95 | 336.34
      10000 | 4.31 | 321.84
      11000 | 4.71 | 307.41
      12000 | 5.14 | 293.26
      13000 | 5.70 | 278.84
      14000 | 6.13 | 265.38
      15000 | 6.52 | 253.05
      16000 | 6.89 | 241.82

      Graphically, the results show that the throughput starts at approx. 200 Objects/Sec,
      climbs to 370 (for 6000 objects), and then linearily
      decreases at approx. 12 Objects/Sec for every 1000 objects
      added.


      Cache Configuration File

      <?xml version="1.0" encoding="UTF-8"?>
      
      <!-- ===================================================================== -->
      <!-- -->
      <!-- Sample PojoCache Service Configuration -->
      <!-- -->
      <!-- ===================================================================== -->
      
      <server>
      
       <!-- Used inside JBoss AS -->
       <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar" />
      
      
       <!-- ==================================================================== -->
       <!-- Defines configuration -->
       <!-- ==================================================================== -->
      
       <mbean code="org.jboss.cache.aop.PojoCache"
       name="jboss.cache:service=testPojoCache">
      
       <!-- Used inside JBoss AS -->
       <depends>jboss:service=Naming</depends>
       <depends>jboss:service=TransactionManager</depends>
      
       <!--
       Configure the TransactionManager
       -->
       <attribute name="TransactionManagerLookupClass">org.jboss.cache.DummyTransactionManagerLookup</attribute>
      
       <!--
       Node locking scheme:
       OPTIMISTIC
       PESSIMISTIC (default)
       -->
       <attribute name="NodeLockingScheme">OPTIMISTIC</attribute>
      
       <!--
       Note that this attribute is IGNORED if your NodeLockingScheme above is OPTIMISTIC.
      
       Isolation level : SERIALIZABLE
       REPEATABLE_READ (default)
       READ_COMMITTED
       READ_UNCOMMITTED
       NONE
       -->
       <attribute name="IsolationLevel">NONE</attribute>
      
       <!--
       Valid modes are LOCAL
       REPL_ASYNC
       REPL_SYNC
       INVALIDATION_ASYNC
       INVALIDATION_SYNC
       -->
       <attribute name="CacheMode">LOCAL</attribute>
      
       <!-- Name of cluster. Needs to be the same for all clusters, in order
       to find each other
       -->
       <attribute name="ClusterName">MVCache-Cluster</attribute>
      
       <!-- JGroups protocol stack properties. Can also be a URL,
       e.g. file:/home/bela/default.xml
       <attribute name="ClusterProperties"></attribute>
       -->
      
       <attribute name="ClusterConfig">
       <config>
       <!-- UDP: if you have a multihomed machine,
       set the bind_addr attribute to the appropriate NIC IP address, e.g bind_addr="192.168.0.2"
       -->
       <!-- UDP: On Windows machines, because of the media sense feature
       being broken with multicast (even after disabling media sense)
       set the loopback attribute to true -->
       <UDP mcast_addr="228.1.2.3" mcast_port="48866" ip_ttl="64" ip_mcast="true" mcast_send_buf_size="150000"
      mcast_recv_buf_size="80000" ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
      loopback="true" />
       <PING timeout="2000" num_initial_members="3" up_thread="false" down_thread="false" />
       <MERGE2 min_interval="10000" max_interval="20000" />
       <!-- <FD shun="true" up_thread="true" down_thread="true" />-->
       <FD_SOCK />
       <VERIFY_SUSPECT timeout="3000" up_thread="false" down_thread="false" />
       <pbcast.NAKACK gc_lag="50" retransmit_timeout="600,1200,2400,4800" max_xmit_size="8192"
      up_thread="false" down_thread="false" />
       <UNICAST timeout="600,1200,2400" window_size="100" min_threshold="10" down_thread="false" />
       <pbcast.STABLE desired_avg_gossip="20000" up_thread="false" down_thread="false" />
       <FRAG frag_size="8192" down_thread="false" up_thread="false" />
       <pbcast.GMS join_timeout="5000" join_retry_timeout="2000" shun="true" print_local_addr="true" />
       <pbcast.STATE_TRANSFER up_thread="true" down_thread="true" />
       </config>
       </attribute>
      
      
       <!--
       Whether or not to fetch state on joining a cluster
       NOTE this used to be called FetchStateOnStartup and has been renamed to be more descriptive.
       -->
       <attribute name="FetchInMemoryState">true</attribute>
      
       <!--
       The max amount of time (in milliseconds) we wait until the
       initial state (ie. the contents of the cache) are retrieved from
       existing members in a clustered environment
       -->
       <attribute name="InitialStateRetrievalTimeout">20000</attribute>
      
       <!--
       Number of milliseconds to wait until all responses for a
       synchronous call have been received.
       -->
       <attribute name="SyncReplTimeout">15000</attribute>
      
       <!-- Max number of milliseconds to wait for a lock acquisition -->
       <attribute name="LockAcquisitionTimeout">10000</attribute>
      
       <!-- Name of the eviction policy class. -->
       <attribute name="EvictionPolicyClass"></attribute>
      
       <!--
       Indicate whether to use region based marshalling or not. Set this to true if you are running under a scoped
       class loader, e.g., inside an application server. Default is "false".
       -->
       <attribute name="UseRegionBasedMarshalling">false</attribute>
       </mbean>
      
      
       <!-- Uncomment to get a graphical view of the TreeCache MBean above -->
       <!-- <mbean code="org.jboss.cache.TreeCacheView" name="jboss.cache:service=TreeCacheView">-->
       <!-- <depends>jboss.cache:service=TreeCache</depends>-->
       <!-- <attribute name="CacheService">jboss.cache:service=TreeCache</attribute>-->
       <!-- </mbean>-->
      
      
      </server>


      Test Code
      package test.stress;
      
      import java.util.Properties;
      
      import javax.naming.Context;
      
      import junit.framework.TestCase;
      
      import org.apache.log4j.Logger;
      import org.jboss.cache.PropertyConfigurator;
      import org.jboss.cache.aop.PojoCache;
      
      /**
       *
       * Tests performance with JBC 1.4.0 SP1. Adding new primitive objects only.
       *
       */
      public class StressTestAdd extends TestCase {
      
       static {
       org.apache.log4j.PropertyConfigurator.configure("resources/test.log4j");
       }
      
       private static final long TEST_TIME_SECONDS = 70;
       private static final String CONFIG_FILENAME = "resources/session-cache/local_cache.xml";
       private static final Logger log = Logger.getLogger(StressTestAdd.class);
      
       public void testOneCache() throws Exception {
      
       // configure cache, don't start it yet
       Properties prop = new Properties();
       prop.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.cache.transaction.DummyContextFactory");
       final PojoCache pojoCache = new PojoCache();
       org.jboss.cache.PropertyConfigurator config = new PropertyConfigurator();
       config.configure(pojoCache, CONFIG_FILENAME);
      
      
       // declare runner thread
       Thread runner = new Thread(new Runnable() {
      
       private int counter = 0;
       boolean running = true;
       private final long start = System.currentTimeMillis();
      
       public void run() {
       try {
       while (running) {
       counter++;
      
       // key and value for attach(key, value);
       String key = "/root/Key-" + counter;
       Object value = "dummy" + counter;
      
       // put in cache
       Object previous = pojoCache.putObject(key, value);
      
       // verify no object was already attached with that key
       if (previous != null) {
       log.error("A previous key existed");
       running = false;
       fail("Attempt to update an existing key " + key);
       }
      
       // display counter every 1000 invocations
       if (counter % 1000 == 0) {
       long elapsed = System.currentTimeMillis() - start; // approx.
       log.info("Number = " + counter + ", after approx. "
       + elapsed / 1000.0 + " seconds.");
       }
       }
       } catch (Exception e) {
       log.error("Exception caught: ", e);
       fail(e.toString());
       }
       }
       });
      
      
       // start stress
       log.info("Stress test about to begin...");
      
       pojoCache.start();
       runner.start();
       Thread.sleep(TEST_TIME_SECONDS * 1000);
      
       log.info("...Stress test ended");
      
       }
      
      }



        1 2 Previous Next