11 Replies Latest reply on Jul 25, 2006 11:41 PM by confuz

    Problem using FileCacheLoader

    bruce_zhang

      Hi,

      I am setting up a prototype to evaluate our re-engineering project.
      I need a cache system to hold the transaction data and I've choose JBoss Cache after browsing the website. So I am new to JBoss Cache and I meet a problem in setting up using the FileCacheLoader.

      My intension is to first setup a simple Cache with file persist for fail over recovery. Everything seems working fine untill I set up a FileCacheLoder.
      The problem is it becomes SO SLOW for putting entries after set up the FileCacheLoader. I have a testing program that simply write and read
      some objects to the same cache node in a loop. Before I set up the FileCacheLoader, it take only about 1-2ms to read/write an object.
      But after using the FileCacheLoader, it takes 100ms to write an entry in the beginning and the time keeps growing and finally reach about 1000ms per write. I know there must be some performance degrade after using cacheLoader but this kind of degrade seems not right . Can somebody take a look at my configuration file as following, Thanks a lot !

      Besides, will a JDBCCacheLoader provide better performance ? It will be greatly appreciated if somebody can provide samples for that .

      =========== jbosscache-config.xml ==========
      
      <server>
      
      <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/>
      
       <mbean code="org.jboss.cache.TreeCache" name="jboss.cache:service=TreeCache">
      
       <depends>jboss:service=Naming</depends>
       <depends>jboss:service=TransactionManager</depends>
      
       <attribute name="TransactionManagerLookupClass">org.jboss.cache.DummyTransactionManagerLookup</attribute>
      
       <attribute name="IsolationLevel">REPEATABLE_READ</attribute>
      
       <attribute name="CacheMode">LOCAL</attribute>
      
       <attribute name="ClusterName">TreeCache-Cluster</attribute>
      
       <attribute name="ClusterConfig">
       <config>
       <!-- UDP: if you have a multihomed machine,
       set the bind_addr attribute to the appropriate NIC IP address
       -->
       <!-- UDP: On Windows machines, because of the media sense feature
       being broken with multicast (even after disabling media sense)
       set the loopback attribute to true
       -->
       <UDP mcast_addr="228.1.2.3" mcast_port="45566" ip_ttl="64" ip_mcast="true"
       mcast_send_buf_size="150000" mcast_recv_buf_size="80000" ucast_send_buf_size="150000"
       ucast_recv_buf_size="80000" loopback="false" />
       <PING timeout="2000" num_initial_members="3" up_thread="false" down_thread="false" />
       <MERGE2 min_interval="10000" max_interval="20000" />
       <FD shun="true" up_thread="true" down_thread="true" />
       <VERIFY_SUSPECT timeout="1500" up_thread="false" down_thread="false" />
       <pbcast.NAKACK gc_lag="50" max_xmit_size="8192" retransmit_timeout="600,1200,2400,4800" up_thread="false"
       down_thread="false" />
       <UNICAST timeout="600,1200,2400" window_size="100" min_threshold="10" down_thread="false" />
       <pbcast.STABLE desired_avg_gossip="20000" up_thread="false" down_thread="false" />
       <FRAG frag_size="8192" down_thread="false" up_thread="false" />
       <pbcast.GMS join_timeout="5000" join_retry_timeout="2000" shun="true" print_local_addr="true" />
       <pbcast.STATE_TRANSFER up_thread="false" down_thread="false" />
       </config>
       </attribute>
      
       <attribute name="SyncReplTimeout">10000</attribute>
      
       <attribute name="LockAcquisitionTimeout">15000</attribute>
      
       <attribute name="EvictionPolicyClass"></attribute>
      
       <attribute name="CacheLoaderConfiguration">
       <config>
       <passivation>false</passivation>
       <preload>/test</preload>
       <shared>false</shared>
      
       <cacheloader>
       <class>org.jboss.cache.loader.FileCacheLoader</class>
       <properties>
       location=/tmp/filetore
       </properties>
       <async>true</async>
      
       <fetchPersistentState>true</fetchPersistentState>
       <ignoreModifications>false</ignoreModifications>
       <purgeOnStartup>false</purgeOnStartup>
       </cacheloader>
      
       </config>
       </attribute>
      
       </mbean>
      </server>



      ===== Testing Program =====
      
      TreeCache cache = new TreeCache();
      PropertyConfigurator pc = new PropertyConfigurator();
      pc.configure(cache, "jbosscache-config.xml");
      
      cache.createService();
      cache.startService();
      
      cache.put("/test", key, obj );
      
      ....




      [img][/img][img][/img]

        • 1. Re: Problem using FileCacheLoader
          manik

          You're right, that kind of performance degradation is strange. Do you have a lot of contention for disk IO on your system or do you have a particularly slow/fragmented disk?

          I'd also look at BDBJE cache loader and JDBM cache loaders as potentially faster alternatives. I'd also upgrade to the latest JBoss Cache 1.4.0 which is much more efficient with cache loaders.

          Cheers,
          Manik

          • 2. Re: Problem using FileCacheLoader
            bruce_zhang

            Thanks for the quick response.

            As of my disk IO. It's on a development machine but I don't feel particular slow because I also use it for JMS queue persist. And I don't see much delay for the JMS Queue processing when I sended persistent msg.

            I will update to 1.4 (current is 1.3) and try BDBJE cache loader and let you know.

            Regards.
            -Bruce

            • 3. Re: Problem using FileCacheLoader
              bruce_zhang

              Hi, I got a 50% performance gain after I upgrade to version 1.4 even still using FileCacheLoader. (That was 100-300 ms per write, still slow though)

              After I switched to JDBM cache loader, I felt that is what I am looking for: the average writing time reduced to 20-30ms and kept stable when I dumped more data.


              Two more questions remain:

              - Do I need JDBM project license if I want to use JdbmCacheLoader in production ?

              - I simply replaced the FileCacheLoader with JdbmCacheLoader in the above config file. With the above setting, should it auto-preload all data under "/test" Node when cache.startService() get called as in my testing problem ? How do I know because it seems not doing that way.

              Thanks a lot!
              -Bruce

              • 4. Re: Problem using FileCacheLoader
                genman

                bruce -

                a lot of the persistence mechanisms (including the file/JDBC) use a single record per Fqn, meaning that if you are adding a lot of keys to an Fqn in separate operations, each one will be increasingly slow.

                The JDBM implementation that I wrote uses a single record per key.

                • 5. Re: Problem using FileCacheLoader
                  genman

                  JDBM is open source so you don't have to worry about licencing.

                  The persistence manager isn't as well tested as the others, unfortunately. Perhaps there was a configuration mistake? What does the log message at start up say? You should see something like:


                  JDBM database /tmp/foo opened with 1242 entries
                  .

                  • 6. Re: Problem using FileCacheLoader
                    bruce_zhang

                    genman:

                    Thank you for the information.
                    Below is the log I got when the cache start up, it seems found all the objects under the node "/loanet" but not loaded in memory:


                    main org.jboss.cache.PropertyConfigurator - Found existing property editor for org.w3c.dom.Element: org.jboss.util.propertyeditor.ElementEditor@21f46a
                    main org.jboss.cache.PropertyConfigurator - attribute size: 10
                    main org.jboss.cache.PropertyConfigurator - setting attribute TransactionManagerLookupClass to org.jboss.cache.DummyTransactionManagerLookup
                    main org.jboss.cache.PropertyConfigurator - setting attribute NodeLockingScheme to OPTIMISTIC
                    main org.jboss.cache.PropertyConfigurator - setting attribute IsolationLevel to REPEATABLE_READ
                    main org.jboss.cache.PropertyConfigurator - setting attribute CacheMode to LOCAL
                    main org.jboss.cache.PropertyConfigurator - setting attribute ClusterName to TreeCache-Cluster
                    main org.jboss.cache.PropertyConfigurator - Invoking setter method: public void org.jboss.cache.TreeCache.setClusterConfig(org.w3c.dom.Element) with parameter "[config: null]" of type class weblogic.apache.xerces.dom.DeferredElementImpl
                    main org.jboss.cache.TreeCache - setting cluster properties from xml to: UDP(ip_mcast=true;ip_ttl=64;loopback=false;mcast_addr=228.1.2.3;mcast_port=45566;mcast_recv_buf_size=80000;mcast_send_buf_size=150000;ucast_recv_buf_size=80000;ucast_send_buf_size=150000):PING(down_thread=false;num_initial_members=3;timeout=2000;up_thread=false):MERGE2(max_interval=20000;min_interval=10000):FD(down_thread=true;shun=true;up_thread=true):VERIFY_SUSPECT(down_thread=false;timeout=1500;up_thread=false):pbcast.NAKACK(down_thread=false;gc_lag=50;max_xmit_size=8192;retransmit_timeout=600,1200,2400,4800;up_thread=false):UNICAST(down_thread=false;min_threshold=10;timeout=600,1200,2400;window_size=100):pbcast.STABLE(desired_avg_gossip=20000;down_thread=false;up_thread=false):FRAG(down_thread=false;frag_size=8192;up_thread=false):pbcast.GMS(join_retry_timeout=2000;join_timeout=5000;print_local_addr=true;shun=true):pbcast.STATE_TRANSFER(down_thread=false;up_thread=false)
                    main org.jboss.cache.PropertyConfigurator - Invoking setter method: public void org.jboss.cache.TreeCache.setSyncReplTimeout(long) with parameter "10000" of type class java.lang.Long
                    main org.jboss.cache.PropertyConfigurator - Invoking setter method: public void org.jboss.cache.TreeCache.setLockAcquisitionTimeout(long) with parameter "15000" of type class java.lang.Long
                    main org.jboss.cache.PropertyConfigurator - getSubElementObject(): element object. Does not exist for config
                    main org.jboss.cache.PropertyConfigurator - setting attribute EvictionPolicyClass to
                    main org.jboss.cache.PropertyConfigurator - Invoking setter method: public void org.jboss.cache.TreeCache.setCacheLoaderConfiguration(org.w3c.dom.Element) with parameter "[config: null]" of type class weblogic.apache.xerces.dom.DeferredElementImpl
                    main org.jboss.cache.transaction.DummyTransactionManager - binding of DummyTransactionManager failed
                    javax.naming.OperationNotSupportedException: bind not allowed in a ReadOnlyContext; remaining name '/TransactionManager'
                     at weblogic.jndi.factories.java.ReadOnlyContextWrapper.newOperationNotSupportedException(ReadOnlyContextWrapper.java:145)
                     at weblogic.jndi.factories.java.ReadOnlyContextWrapper.newOperationNotSupportedException(ReadOnlyContextWrapper.java:161)
                     at weblogic.jndi.factories.java.ReadOnlyContextWrapper.bind(ReadOnlyContextWrapper.java:57)
                     at weblogic.jndi.internal.AbstractURLContext.bind(AbstractURLContext.java:45)
                     at javax.naming.InitialContext.bind(InitialContext.java:355)
                     at org.jboss.cache.transaction.DummyTransactionManager.getInstance(DummyTransactionManager.java:33)
                     at org.jboss.cache.DummyTransactionManagerLookup.getTransactionManager(DummyTransactionManagerLookup.java:17)
                     at org.jboss.cache.TreeCache._createService(TreeCache.java:1311)
                     at org.jboss.cache.TreeCache.createService(TreeCache.java:1297)
                     at com.citigroup.posmgmt.biz.JBossCacheImpl.startService(JBossCacheImpl.java:54)
                     at com.citigroup.posmgmt.biz.JBossCacheImpl.startup(JBossCacheImpl.java:91)
                     at weblogic.t3.srvr.StartupClassService.invokeStartup(StartupClassService.java:177)
                     at weblogic.t3.srvr.StartupClassService.invokeClass(StartupClassService.java:158)
                     at weblogic.t3.srvr.StartupClassService.access$000(StartupClassService.java:36)
                     at weblogic.t3.srvr.StartupClassService$1.run(StartupClassService.java:121)
                     at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
                     at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
                     at weblogic.t3.srvr.StartupClassService.invokeStartupClass(StartupClassService.java:116)
                     at weblogic.t3.srvr.PostDeploymentStartupService.resume(PostDeploymentStartupService.java:63)
                     at weblogic.t3.srvr.SubsystemManager.resume(SubsystemManager.java:136)
                     at weblogic.t3.srvr.T3Srvr.resume(T3Srvr.java:965)
                     at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:360)
                     at weblogic.Server.main(Server.java:32)
                    main org.jboss.cache.loader.AsyncCacheLoader - setConfig {location=/tmp/cacheStore}
                    main org.jboss.cache.loader.jdbm.JdbmCacheLoader - Configuring cache loader with location = /tmp/cacheStore
                    main org.jboss.cache.TreeCache - Not using an EvictionPolicy
                    main org.jboss.cache.factories.InterceptorChainFactory - interceptor chain is:
                    class org.jboss.cache.interceptors.CallInterceptor
                    class org.jboss.cache.interceptors.OptimisticNodeInterceptor
                    class org.jboss.cache.interceptors.OptimisticCreateIfNotExistsInterceptor
                    class org.jboss.cache.interceptors.OptimisticValidatorInterceptor
                    class org.jboss.cache.interceptors.OptimisticLockingInterceptor
                    class org.jboss.cache.interceptors.CacheStoreInterceptor
                    class org.jboss.cache.interceptors.CacheLoaderInterceptor
                    class org.jboss.cache.interceptors.TxInterceptor
                    class org.jboss.cache.interceptors.CacheMgmtInterceptor
                    main org.jboss.cache.TreeCache - cache mode is local, will not create the channel
                    main org.jboss.cache.loader.AsyncCacheLoader - Async cache loader starting: org.jboss.cache.loader.AsyncExtendedCacheLoader@5e25f3 delegate=[JdbmCacheLoader locationStr=/tmp/cacheStore size=-1] processor=null stopped=true batchSize=100 pollWait=100 returnOld=true asyncPut=true queue.capacity()=10000 queue.peek()=null
                    main org.jboss.cache.loader.jdbm.JdbmCacheLoader - Starting JdbmCacheLoader instance.
                    main org.jboss.cache.loader.jdbm.JdbmCacheLoader - JdbmCacheLoader located as 131100
                    main org.jboss.cache.loader.jdbm.JdbmCacheLoader - JDBM database /tmp/cacheStore/TreeCache-Cluster opened with 11134 entries
                    AsyncCacheLoader-1 org.jboss.cache.loader.AsyncCacheLoader - run0
                    main org.jboss.cache.loader.CacheLoaderManager - preloading transient state from cache loader org.jboss.cache.loader.AsyncExtendedCacheLoader@5e25f3 delegate=[JdbmCacheLoader locationStr=/tmp/cacheStore size=11134] processor=TQ t=Thread[AsyncCacheLoader-1,5,main] stopped=false batchSize=100 pollWait=100 returnOld=true asyncPut=true queue.capacity()=10000 queue.peek()=null
                    main org.jboss.cache.loader.CacheLoaderManager - preloading /loanet
                    main org.jboss.cache.interceptors.TxInterceptor - (null) call on method [_get(/loanet, bla, true)]
                    main org.jboss.cache.interceptors.TxInterceptor - Creating transaction for thread Thread[main,5,main]
                    main org.jboss.cache.interceptors.TxInterceptor - local transaction exists - registering global tx if not present for Thread[main,5,main]
                    main org.jboss.cache.interceptors.TxInterceptor - Associated gtx in txTable is null
                    main org.jboss.cache.TreeCache - created new GTX: GlobalTransaction:<null>:1, local TX=org.jboss.cache.transaction.DummyTransaction@562791
                    main org.jboss.cache.interceptors.TxInterceptor - Registering sync handler for tx org.jboss.cache.transaction.DummyTransaction@562791, gtx GlobalTransaction:<null>:1
                    main org.jboss.cache.transaction.DummyTransaction - registering synchronization handler tx=org.jboss.cache.transaction.DummyTransaction@562791, handlers=[]
                    main org.jboss.cache.interceptors.TxInterceptor - registering for TX completion: SynchronizationHandler(TxInterceptor.LocalSynchronizationHandler(gtx=GlobalTransaction:<null>:1, tx=org.jboss.cache.transaction.DummyTransaction@562791))
                    main org.jboss.cache.interceptors.CacheLoaderInterceptor - invoke _get(/loanet, bla, true)
                    main org.jboss.cache.interceptors.CacheLoaderInterceptor - load element /loanet mustLoad=true
                    main org.jboss.cache.interceptors.CacheLoaderInterceptor - loadNode /loanet
                    main org.jboss.cache.loader.jdbm.JdbmCacheLoader - get /loanet map={600001466=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@4a1bda, 600003106=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@16a93b8, 600007439=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@838a2, 600010984=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@1be40d5, 600007195=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@19261de, 600006550=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@157566d, 600003453=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@eb203b, 600005005=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@911fb5, 600006685=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@1b8b5cd, 600006145=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@1a7d9e7, 600004544=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@d2ee5d, 600004711=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@1f568f, 600002342=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@16c02df, 600002640=com.citigroup.posmgmt.vo.LoaNetActivityTbEx@30803a, 600
                    .......
                    [-- More data info for the node --]
                    
                    main org.jboss.cache.interceptors.CacheLoaderInterceptor - nodeExists true
                    main org.jboss.cache.Node - created child: fqn=/loanet
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - CacheStoreInterceptor called with meth _get(/loanet, bla, true)
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - transactional so don't put stuff in the cloader yet.
                    main org.jboss.cache.transaction.DummyTransaction - processing beforeCompletion for tx=org.jboss.cache.transaction.DummyTransaction@562791, handlers=[TxInterceptor.LocalSynchronizationHandler(gtx=GlobalTransaction:<null>:1, tx=org.jboss.cache.transaction.DummyTransaction@562791)]
                    main org.jboss.cache.interceptors.TxInterceptor - Running beforeCompletion on gtx GlobalTransaction:<null>:1
                    main org.jboss.cache.interceptors.TxInterceptor - No modifications in this tx. Skipping beforeCompletion()
                    main org.jboss.cache.transaction.DummyTransaction - processing afterCompletion for tx=org.jboss.cache.transaction.DummyTransaction@562791, handlers=[TxInterceptor.LocalSynchronizationHandler(gtx=GlobalTransaction:<null>:1, tx=org.jboss.cache.transaction.DummyTransaction@562791)]
                    main org.jboss.cache.interceptors.TxInterceptor - calling aftercompletion for GlobalTransaction:<null>:1
                    main org.jboss.cache.interceptors.TxInterceptor - Running commit phase. One phase? false
                    main org.jboss.cache.interceptors.TxInterceptor - running commit for GlobalTransaction:<null>:1
                    main org.jboss.cache.interceptors.CacheLoaderInterceptor - invoke commit(GlobalTransaction:<null>:1)
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - CacheStoreInterceptor called with meth commit(GlobalTransaction:<null>:1)
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - transactional so don't put stuff in the cloader yet.
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - Commit called with no modifications; ignoring.
                    main org.jboss.cache.interceptors.OptimisticValidatorInterceptor - commiting validated changes
                    main org.jboss.cache.interceptors.OptimisticValidatorInterceptor - Merging node /loanet not necessary since the node is not dirty
                    main org.jboss.cache.interceptors.TxInterceptor - Finished local commit/rollback method for GlobalTransaction:<null>:1
                    main org.jboss.cache.interceptors.TxInterceptor - Finished commit phase
                    main org.jboss.cache.loader.jdbm.JdbmCacheLoader - getChildrenNames /loanet
                    main org.jboss.cache.loader.CacheLoaderManager - preloading transient state from cache loader was successful (in 7169 milliseconds)
                    main org.jboss.cache.interceptors.TxInterceptor - (null) call on method [_get(/loanet, bla, true)]
                    main org.jboss.cache.interceptors.TxInterceptor - Creating transaction for thread Thread[main,5,main]
                    main org.jboss.cache.interceptors.TxInterceptor - local transaction exists - registering global tx if not present for Thread[main,5,main]
                    main org.jboss.cache.interceptors.TxInterceptor - Associated gtx in txTable is null
                    main org.jboss.cache.TreeCache - created new GTX: GlobalTransaction:<null>:2, local TX=org.jboss.cache.transaction.DummyTransaction@58ba4b
                    main org.jboss.cache.interceptors.TxInterceptor - Registering sync handler for tx org.jboss.cache.transaction.DummyTransaction@58ba4b, gtx GlobalTransaction:<null>:2
                    main org.jboss.cache.transaction.DummyTransaction - registering synchronization handler tx=org.jboss.cache.transaction.DummyTransaction@58ba4b, handlers=[]
                    main org.jboss.cache.interceptors.TxInterceptor - registering for TX completion: SynchronizationHandler(TxInterceptor.LocalSynchronizationHandler(gtx=GlobalTransaction:<null>:2, tx=org.jboss.cache.transaction.DummyTransaction@58ba4b))
                    main org.jboss.cache.interceptors.CacheLoaderInterceptor - invoke _get(/loanet, bla, true)
                    main org.jboss.cache.interceptors.CacheLoaderInterceptor - load element /loanet mustLoad=false
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - CacheStoreInterceptor called with meth _get(/loanet, bla, true)
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - transactional so don't put stuff in the cloader yet.
                    main org.jboss.cache.transaction.DummyTransaction - processing beforeCompletion for tx=org.jboss.cache.transaction.DummyTransaction@58ba4b, handlers=[TxInterceptor.LocalSynchronizationHandler(gtx=GlobalTransaction:<null>:2, tx=org.jboss.cache.transaction.DummyTransaction@58ba4b)]
                    main org.jboss.cache.interceptors.TxInterceptor - Running beforeCompletion on gtx GlobalTransaction:<null>:2
                    main org.jboss.cache.interceptors.TxInterceptor - No modifications in this tx. Skipping beforeCompletion()
                    main org.jboss.cache.transaction.DummyTransaction - processing afterCompletion for tx=org.jboss.cache.transaction.DummyTransaction@58ba4b, handlers=[TxInterceptor.LocalSynchronizationHandler(gtx=GlobalTransaction:<null>:2, tx=org.jboss.cache.transaction.DummyTransaction@58ba4b)]
                    main org.jboss.cache.interceptors.TxInterceptor - calling aftercompletion for GlobalTransaction:<null>:2
                    main org.jboss.cache.interceptors.TxInterceptor - Running commit phase. One phase? false
                    main org.jboss.cache.interceptors.TxInterceptor - running commit for GlobalTransaction:<null>:2
                    main org.jboss.cache.interceptors.CacheLoaderInterceptor - invoke commit(GlobalTransaction:<null>:2)
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - CacheStoreInterceptor called with meth commit(GlobalTransaction:<null>:2)
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - transactional so don't put stuff in the cloader yet.
                    main org.jboss.cache.interceptors.CacheStoreInterceptor - Commit called with no modifications; ignoring.
                    main org.jboss.cache.interceptors.OptimisticValidatorInterceptor - commiting validated changes
                    main org.jboss.cache.interceptors.OptimisticValidatorInterceptor - Merging node /loanet not necessary since the node is not dirty
                    main org.jboss.cache.interceptors.TxInterceptor - Finished local commit/rollback method for GlobalTransaction:<null>:2
                    main org.jboss.cache.interceptors.TxInterceptor - Finished commit phase
                    main org.jboss.cache.loader.jdbm.JdbmCacheLoader - getChildrenNames /loanet



                    I have also tried to change to JBossTransactionManagerLookup but it totally not working (I am using Weblogic 8.1 AS). Did I miss anything in the configuration ??

                    Thanks a lot !
                    -Bruce




                    • 7. Re: Problem using FileCacheLoader
                      bruce_zhang

                      genman:

                      Sorry, I missed the trace in the log file:

                      main org.jboss.cache.loader.jdbm.JdbmCacheLoader - JDBM database /tmp/cacheStore/TreeCache-Cluster opened with 11134 entries.


                      Does that mean all the 11134 entries had been Physically loaded in memory? It seemed only take 7 seconds -- was the time reasonable ?
                      The reason for the doubt was also because the GET performance SLOWED down pretty much after I restart the Cache. The test case was as following:

                      1. In a for loop, right after I PUT an entry, I call a GET on the same node with same key to make sure the entry saved successfully.
                      2. In step 1 each GET took average time about 5-10 ms.
                      3. After PUT about 11K entries (diff key/value under one node). I stop the service and restart the Cache (currently it hosted on Weblogic AS so I acturally restarted the Weblogic AS). And I hope the cache would pre-load the 11K entries in memory from the persistent file in the startup.
                      4. Then I GET the entry again. But this time it was much slower -- each took about 70-100ms. So it seemed to me the object may not really found in the memory.

                      Could you please take a look for the setting and log, and check whether I missed anything ?

                      Besides, in our current business requirement, we actually need to write/read a lot data to cache in a Burst mode, so we pretty concern about the cache write/read performance but not care too much how it persist. When write to cache most of the data are new, and when read from the cache if the data not in Cache memory it won't be in the file either (We prefer cache all data in mem without expiration). Could you please suggest the best configuration for it ?

                      Also considering the tree structure, if I store the data under multiple node instead one, will that improve the read/write performance ?

                      Thank you very much !
                      -Bruce






                      • 8. Re: Problem using FileCacheLoader
                        genman


                        Sorry to confuse you, it says the number of entries in the file were opened. I can assure you that none of those were loaded into memory before your first TreeCache.get(). I don't think it will "preload" the data unless you tell it to. I wonder if there's a option to do it in the current release?

                        My suggestion for improving application performance would be to use the Async cache configuration. It will give you much better performance when doing multiple "put" commands.

                        • 9. Re: Problem using FileCacheLoader
                          genman


                          In terms of design, it's a good idea to make each tree node represent a relational database row (fixed in size), rather than store the contents of an entire database table.

                          For example, if you were storing customers, each customer should have its own node.

                          • 10. Re: Problem using FileCacheLoader
                            bruce_zhang

                            You are right! After I changed the data structure stored in the cache as you suggested, it gained great performance improve in read/write and can fully meet my current requirement. Looks like how to store date is also a key when using TreeCache.

                            Thank you so much for all the help! You guys are doing amazing jobs !

                            -Bruce

                            • 11. Re: Problem using FileCacheLoader
                              confuz

                               

                              "genman" wrote:

                              In terms of design, it's a good idea to make each tree node represent a relational database row (fixed in size), rather than store the contents of an entire database table.

                              For example, if you were storing customers, each customer should have its own node.


                              Hi Genman,
                              I can not found any passivation policy support node's value level. but what i need is 100,000 in one node, 20,000 in-memory and 80,000 in passivated status. so i implement that by what you suggested(one node represent a record), but currently in my local machine, the program will hang when putting nodes in to cache and nodes had exceeded 40,000
                              any suggestions on this issue, thanks

                              regards