4 Replies Latest reply on Feb 20, 2008 5:47 AM by mircea.markus

    Cache Persistence granularity

    debbanerjee

      We are using jboss cache(version: jbosscache-core-2.1.0.CR2, tree cache) within a Java service. Our cache has very few nodes (around 3). Each of those nodes can have lots of data (>100K entries). We are using BDB-J for cache persistence, using the BDBJECacheLoader. Our data is non-transactional: we are caching Directory (LDAP server) data. So we are runinng with isolation level none.

      Reviewing the documentation, and looking at the test results the following seems to be true:

      When I update a map entry in a node then the entire map is written to disk. I had expected only that specific entry would be written out to disk. Clearly this will not satisfy our performance requirements since a map entry update would trigger a super-large disk write.

      Can this be confirmed? If so are there workarounds where only that single entry is updated in the above use case? Do I need to write a custom CacheLoader instance to get this to behave performantly?

      Thanks
      --Deb

        • 1. Re: Cache Persistence granularity
          lovelyliatroim

           


          If so are there workarounds where only that single entry is updated in the above use case?


          Can you change your entries in your map into a node? This would solve your problem.

          e.g
          You have a path like so
          /a/b/c and node "c" contains 100 elements in your hashmap. Now if you changed the path to be
          a/b/c/c1
          a/b/c/c2
          a/b/c/c3

          And c1,c2 and c3 were originally hash entries the old way but now you have made them into nodes and each node will now only contain 1 element in the hashmap then this would solve your update problem.




          • 2. Re: Cache Persistence granularity
            debbanerjee

            Besides persistence, are there any other considerations in organizing the cache as a small number of nodes with large number of entries per node, or with a large number of nodes (and little data per node)? My initial assumption is that Nodes are heavier-weight objects and, therefore, less is more.

            Comments/opinions welcome.

            • 3. Re: Cache Persistence granularity
              debbanerjee


              Once I redesigned the cache to have lots of Nodes with one value per node (instead of a few cache nodes with lots of values) the persistence performance improved dramatically. The data set size is around 100K entries;

              using
              1. file persistence my test run duration went down from 90 min to 2.5 min.
              2. using BDB-J persistence duration went from down to 30 secs.

              Great. So far I haven't found any negatives using this design.

              --Deb

              • 4. Re: Cache Persistence granularity
                mircea.markus

                Thanks for the feedback, looks like an thing that we should definitely look for optimizations. I've added to http://jira.jboss.com/jira/browse/JBCACHE-1221