8 Replies Latest reply on Jun 15, 2012 12:31 PM by dex80526

    Needs a way to have all cache entries stored in cache loader but only partially loaded in memory

    dex80526

      Right now, Infinispan supports passivation and eviction to manager how cache entries stored (in cach store or memory).

      Passivation if enabled, it will allow to remove cache entries from memory and store in cache stores. But, the cache store have only subset of data.

       

      In our case, we need to have cache store (for persistent purpose) to store all cache data all the time, and have only part of cache entries (spefified by either maxIdle or lifetime, or max number) loaded in memory.

       

      I do not see any way today Infinispan allows me to do this without have another external data store.

       

      If I use eviction to specify max number entries, it will remove the data from both cache store and memory.

       

      Any suggestions?

        • 1. Re: Needs a way to have all cache entries stored in cache loader but only partially loaded in memory
          mgencur

          There is one option which is close to what you need.

          When you specify passivation == off and eviction == on, the cache store will be basically a superset of the cache. Every time you store someting in the cache, the cache entry will go also to the cache store. Furthermore, when eviction is enabled and you reached maxEntries, some of the entries (according to the strategy in use) will be removed from the cache but not from the cache store (don't get it confused with expiration).

           

          Later, when you call get() for an entry that is only in the cache store, it will be loaded into the memory but won't be removed from the cache store.

          • 2. Re: Needs a way to have all cache entries stored in cache loader but only partially loaded in memory
            dex80526

            Martin: I tested that configuration (passivation=false and envition enabled with maxEntries specified), and found the evition removes the cache entries from both the cache and cache store.  Maybe my cinfiguration is not right?

             

               <namedCache name="keychain" >

                  <clustering mode="replication">

                     <stateTransfer

                        timeout="240000"

                        fetchInMemoryState="true"

                        fetchPersistentState="false"

                     />

                   

                     <async useReplQueue="true" replQueueInterval="500" replQueueMaxElements="100" asyncMarshalling="false" />

                   

                  </clustering>

                  <transaction transactionMode="TRANSACTIONAL" />

                 <eviction>

                    maxEntries="100"

                   strategy="LRU"

                 </eviction>  

                   <expiration

                    wakeUpInterval="100"

                     lifespan="-1"

                     maxIdle="-1"

                  />      

                  <loaders

                        passivation="false"

                        shared="false"

                        preload="false">

                       <loader

                          class="org.infinispan.loaders.jdbc.stringbased.JdbcStringBasedCacheStore"

                          fetchPersistentState="true"

                          purgeOnStartup="false">

             

                          <properties>

                            <property name="stringsTableNamePrefix" value="ISPN_STRING_TABLE"/>

                            <property name="idColumnName" value="ID_COLUMN"/>

                            <property name="dataColumnName" value="DATA_COLUMN"/>

                            <property name="timestampColumnName" value="TIMESTAMP_COLUMN"/>

                            <property name="timestampColumnType" value="BIGINT"/>

                            <property name="connectionFactoryClass"

                                            value="org.infinispan.loaders.jdbc.connectionfactory.ManagedConnectionFactory"/>

                            <property name="datasourceJndiLocation" value="java:comp/env/jdbc/UserProfileDB"/>

             

                            <property name="idColumnType" value="VARCHAR(255)"/>

                            <property name="dataColumnType" value="BLOB"/>

                            <property name="dropTableOnExit" value="false"/>

                            <property name="createTableOnStart" value="true"/>

                         </properties>

                             <async enabled="true" flushLockTimeout="15000" shutdownTimeout="30000" modificationQueueSize="1000" threadPoolSize="100"/>

                       </loader>

             

                 </loaders>

               </namedCache>

            • 3. Re: Needs a way to have all cache entries stored in cache loader but only partially loaded in memory
              mgencur

              Hi Dex,

              your configuration seems to be correct. However, I'm pretty sure this is working, or at least it was working a short time ago. Could you please file a JIRA and attach a TRACE log? If it is not working, it's most likely a bug.

              • 5. Re: Needs a way to have all cache entries stored in cache loader but only partially loaded in memory
                dex80526

                Okay, I'll try it again and let you know the results.

                • 6. Re: Needs a way to have all cache entries stored in cache loader but only partially loaded in memory
                  dex80526

                  I tried some simple tests. The eviction works as expected, i.e., eviction removes entries only from the memory not the store. The only minor issue is that the actual max entries in memory is quite off to the maxEntries I speciped.   That's what I am looking for. Thanks.

                  • 7. Re: Needs a way to have all cache entries stored in cache loader but only partially loaded in memory
                    mgencur

                    Dex,

                    the maxEntries parameter should be a power of 2 (2,4,8,16,...), when you set a number which is not a power of 2, it will be rounded to the closess higher number which is power of two. However, you will rarely see exactly that number of entries in the cache. The rule is that the number of entries in the cache must not be higher than maxEntries, usually it will be lower than that.

                    • 8. Re: Needs a way to have all cache entries stored in cache loader but only partially loaded in memory
                      dex80526

                      Martn, thanks for the info on maxEntries.