3 Replies Latest reply on Jan 9, 2012 9:05 AM by Randall Hauch

    Hangs and poor performance using JpaSource

    Duncan McIntyre Newbie

      Hi,

       

      I am in the process of running performance tests to evaluate different JCR implementations. To do this I have ported the jackrabbit-jcr-benchmark package from the Jackrabbit project so that I can build it against Jackrabbit 1.6 and 2.3 as well as Modeshape.

       

      All the tests run on Modeshape with the Memory connector, but most of them hang when using Jpa (Hsqldb or Mysql). I'm not quite sure what is happening, but it looks like there is some kind of infinite loop during session.save(). Thes tests have been run against 2.6.0-FINAL, 2.7.0-FINAL and the current 2.8.0-SNAPSHOT (ModeShape-modeshape-4110d75).

       

      I have attached full sources.

       

      In the class below you'll see from the comment in runTest() how to make this test succeed - otherwise it hangs in afterSuite().

       

      To run the tests, build the bits you want (i.e. jcr-perf-suite and jcr-perf-modeshape) and run com.mom.jcrperf.TestRunner. Example configurations are in jcr-perf-modeshape/config.

       

      At the end of this post you will also find two sets of results: one for Jackrabbit 1.5.7 (a slightly wacky custom build) running against a MySql database in Amazon, the other Modeshape 2.7.0-FINAL with an in-memory connector. As you can see, the results are fairly comparable. Results using the Jpa connector (once the tests have been modified to allow them to run) are off by orders of magnitude. Is there something broken in my configuration, or is Modeshape really this much slower than Jackrabbit?

       

      Thanks for any help,

       

      Duncan

       

       

      public class TransientSomeChildNodesTest extends PerformanceTest {
      
      
          private static final int CHILD_COUNT = 100;
      
      
          private static final int ITERATIONS = 100;
      
      
          private Session session;
      
      
          private Node node;
      
      
          public void beforeSuite() throws RepositoryException {
              session = getRepository().login(getCredentials());
              node = session.getRootNode().addNode("testnode", "nt:unstructured");
              for (int i = 0; i < CHILD_COUNT; i++) {
                  node.addNode("node" + i, "nt:unstructured");
              }
          }
      
      
          public void beforeTest() throws RepositoryException {
          }
      
      
          public void runTest() throws Exception {
              for (int i = 0; i < ITERATIONS; i++) {
                  node.addNode("onemore", "nt:unstructured").remove();
              }
              // Test will run to completion on Modeshape if the following line is uncommented 
              // (but then it's not a Transient test..)
              // session.save();
          }
      
      
          public void afterTest() throws RepositoryException {
          }
      
      
          public void afterSuite() throws RepositoryException {
                    session.getRootNode().getNode("testnode").remove();
                    // Test will hang here when running on Modeshape
                    session.save();
                    session.logout();
          }
      
      
      }
      
      

       

       

      Test environment
        JCR: Jackrabbit version 1.5.7 by Apache Software Foundation
          Persistence Manager Configuration
                schema:mysql
                driver:com.mysql.jdbc.Driver
                bundleCacheSize:2
                schemaObjectPrefix:version_
        JRE: 1.6.0_20 by Sun Microsystems Inc.
        JVM: OpenJDK 64-Bit Server VM version 19.0-b09 by Sun Microsystems Inc.
         OS: Linux version 2.6.35.14-97.44.amzn1.x86_64 for amd64
      
      Test case                               Mean    Sdev     Min     Max   Count
      ----------------------------------------------------------------------------
      1000 x login()                           100     149      39     656      27
      1000 x login().logout()                  108     156      45     635      49
      1 000 000 x refresh(false)               150      28     104     172      34
      ConcurrentReadTest                       431     158     305     825      12
      ConcurrentReadWriteTest                  667     435     316    1418       8
      CreateManyChildNodesTest               14772       0   14772   14772       1
      TransientManyChildNodesTest               27      20      18      79     189
      UpdateManyChildNodesTest                 100      78      49     333      25
      VersionedTreeTest                       6028       0    6028    6028       1
      VersionedTreeTest                       7126       0    7126    7126       1
      SimpleSearchTest                          12      14       8      70     418
      1000 x read a 10kB file                 1733      53    1695    1794       3
      100 x write a 10kB file                 1250     156    1046    1405       5
      read a 100MB file                          1       2       0      62    4475
      write a 100MB file                      3762      30    3740    3783       2
      
      
      Modeshape/memory
      Test case                               Mean    Sdev     Min     Max   Count
      ----------------------------------------------------------------------------
      1000 x login()                            83      18      70     140      59
      1000 x login().logout()                   77      12      70     129      65
      1 000 000 x refresh(false)                87       1      86      90      58
      ConcurrentReadTest                        97      33      70     227      52
      ConcurrentReadWriteTest                  343     607     132    2519      15
      CreateManyChildNodesTest               18224       0   18224   18224       1
      TransientManyChildNodesTest               50      10      37      88     102
      TransientSomeChildNodesTest               37       5      35      67     135
      UpdateManyChildNodesTest                 415       8     404     423       4
      VersionedTreeTest                       5190     717    4683    5697       2
      VersionedBundleTest                    11734       0   11734   11734       1
      SimpleSearchTest                         233       7     228     249      22
      1000 x read a 10kB file                   14       2      13      31     345
      100 x write a 10kB file                   37       1      35      46      90
      read a 100MB file                         37       1      36      38     137
      write a 100MB file                      1337     126    1240    1513       4
      
      
        • 1. Re: Hangs and poor performance using JpaSource
          Randall Hauch Master

          If you haven't already, take a look at How To Select The Right Connectors, which includes suggestions on setting up a 2nd level cache. IMO, that's pretty much required.

           

          Note that the JPA connector will no longer be used in the 3.x codebase (see Next Generation ModeShape), and that we'll provide a way to migrate from 2.x stores to 3.0.

          • 2. Re: Hangs and poor performance using JpaSource
            Duncan McIntyre Newbie

            I already have a cache configured - see below. Is there anything else I should be looking at?

             

            I see that you have a performance test suite in Github, so I'll try building and running that. Do you have a result set you could share showing how Modeshape 2.x compares to Jackrabbit? I'd like to know what sort of numbers you think I should be expecting to see (using MySQL for persistence).

             

            When do you expect to be able to release beta and production versions of 3.x?

             

             

                <mode:sources jcr:primaryType="nt:unstructured">

                    <mode:source jcr:name="Mysql" mode:classname="org.modeshape.connector.store.jpa.JpaSource"

                              mode:model="Simple"

                              mode:dialect="org.hibernate.dialect.MySQL5InnoDBDialect"

                              mode:driverClassName="com.mysql.jdbc.Driver"

                              mode:username="mtmuser"

                              mode:password="medic1"

                              mode:url="jdbc:mysql://__DBSERVER__/modeshape"

                              mode:maximumConnectionsInPool="1"

                              mode:referentialIntegrityEnforced="true"

                              mode:largeValueSizeInBytes="10000"

                              mode:retryLimit="3"

                              mode:compressData="false"

                              mode:predefinedWorkspaceNames="workspace1"

                              mode:showSql="false"

                        mode:autoGenerateSchema="create"

                        mode:creatingWorkspacesAllowed="true"

                              mode:defaultWorkspaceName="workspace1"

                              mode:cacheProviderClassName="org.hibernate.cache.HashtableCacheProvider"

                              mode:readDepth="5"

                              mode:queryIndexesUpdatedSynchronously="false"

                              />

                 </mode:sources>

            • 3. Re: Hangs and poor performance using JpaSource
              Randall Hauch Master

              You might try the Infinispan cache provider, which is faster. See also How To Tune ModeShape for Better Performance

               

              We don't have performance comparisons available, but there are several performance metrics you can run with your own configuration, hardware, and network:

              1. For 2.x, we have a connector benchmarking module (see https://github.com/ModeShape/modeshape/tree/master/utils/modeshape-connector-benchmark) that you can easily run once you've locally done a full build of the codebase. And it's pretty easy to configure.
              2. We have recently started work on a performance suite that is based on the same Jackrabbit's suite you mentioned above, but at this time it's no different than what you've probably already run.

               

              Generally speaking, the JPA connector is not our fastest connector; the Infinispan and disk connectors are significantly faster.