8 Replies Latest reply on Dec 11, 2012 10:50 PM by sannsims

    Updating cache on other nodes

    sannsims Newbie

      Good day,

      I'm sure I've somehow miss it, but I'm looking for documentation/sample code how to update the cache on another node?

       

      Unless I'm doing something wrong, defining "replication" in the configuration file does not automatically update all cache on all nodes?  When my first rest server executes a method to get data, it first checks the cache.  If cache is null, data is retrieved from the database.  When my second rest server executes the same method as the first server to get data, I would expect it to get the data from cache, however it's getting it from the database.  So, it would seem, I need to be able to update the cache on the other server programmatically or can I set this up declaratively?

       

      <clustering mode="replication">   
         <stateTransfer timeout="240000" fetchInMemoryState="false"/> 
         <sync replTimeout="20000"/>
       </clustering>
      

       

      I'm using Infinispan version 5.1, JBoss 5.1, JDK 1.6. 

       

      Thanks for your time and help!

        • 1. Re: Updating cache on other nodes
          Vladimir Blagojevic Master

          No, values should be on both caches (for replication) but it seems that you have to invoke cache put in case of cache miss.

          • 2. Re: Updating cache on other nodes
            sannsims Newbie

            Yes, okay.  I added a put method, the first time I invoke the cache, it is empty...no problem.  I get data from database and then add data to cache using "put".  PROBLEM:  However any subsequent calls to the cache it is empty.

             

            Here's the snippet of code:

             

            public class DemoCacheManager {

             

                 DefaultCacheManager mgr = null;

                 public DemoCacheManager() throws IOException {

                      mgr = new DefaultCacheManager( "infinispan.xml" );

                 }

             

                 public List<Pet> buildPetCache() {

                      //DefaultCacheManager mgr = new DefaultCacheManager( "infinispan.xml" );

                      ArrayList<Pet> petList = new ArrayList<Pet>();

                      Cache<Integer, List<Pet>> cache = mgr.getCache( "pets" );

                      System.out.println( "Is cache empty " + cache.isEmpty() );   //CACHE IS EMPTY - FIRST TIME IN IT SHOULD BE EMPT

                      System.out.println( "KeySet size: " + cache.keySet().size() );

                      if( cache.isEmpty() )

                      {

                           petList = dao.getAllPets();  //getting data from database

                           cache.put( new Integer( 0 ), petList );

                      }

                      System.out.println( "Is cache empty " + cache.isEmpty() );  //NOW CACHE IS NOT EMPTY

                      return petList;

                 }

            }

             

            I'm accessing this code via a Spring bean in my Broker class.

             

            Here's the output from the console:

            16:31:06,625 INFO  [STDOUT]  Is cache empty true

            16:31:06,625 INFO  [STDOUT]  KeySet size: 0

            16:31:06,625 INFO  [STDOUT]  Is cache empty false

             

            16:31:11,531 INFO  [STDOUT]  Is cache empty true

            16:31:11,531 INFO  [STDOUT]  KeySet size: 0

            16:31:11,531 INFO  [STDOUT]  Is cache empty false

             

            16:31:15,267 INFO  [STDOUT]  Is cache empty true

            16:31:15,267 INFO  [STDOUT]  KeySet size: 0

            16:31:15,267 INFO  [STDOUT]  Is cache empty false

             

            I've tried several different ways to get this to work, but I must be missing something???

             

            Thanks!

            • 3. Re: Updating cache on other nodes
              Tristan Tarrant Master

              sannsims,

               

              are you sure pets is clustered ? Can you attach your infinispan.xml ?

               

              cache.getCacheConfiguration().clustering().cacheModeString();

               

              Tristan

              • 4. Re: Updating cache on other nodes
                sannsims Newbie

                Here is the infinispan.xml I am using:

                 

                <?xml version="1.0" encoding="UTF-8"?>
                <infinispan
                      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                      xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"
                      xmlns="urn:infinispan:config:5.1">
                <global>
                  <!-- Enables collection of global JMX statistics, and allows you to customize the JMX domain name under which
                    MBeans are registered. -->
                  <globalJmxStatistics
                             enabled="true"
                             jmxDomain="org.infinispan"
                             allowDuplicateDomains="true"
                             cacheManagerName="SampleCacheManager"/>
                  <asyncListenerExecutor factory="org.infinispan.executors.DefaultExecutorFactory">
                   <properties>
                    <property name="maxThreads" value="5"/>
                    <property name="threadNamePrefix" value="AsyncListenerThread"/>
                   </properties>
                  </asyncListenerExecutor>
                  <!--  If the transport is omitted, there is no way to create distributed or clustered caches.  There is no added cost to
                    defining a transport and not creating a cache that uses one, since the transport is created and initialized lazily. 
                    By default, the JGroupsTransport is used.
                    Note that the JGroups transport uses sensible defaults if no configuration property is defined.
                    See:  http://community.jboss.org/wiki/ClusteredConfigurationQuickStart
                     http://docs.jboss.org/infinispan/5.0/apidocs/org/infinispan/remoting/transport/jgroups/JGroupsTransport.html
                  -->
                  <transport
                   clusterName="infinispan-cluster"
                   machineId="m1"
                   rackId="r1" nodeName="Node-A">
                   <!-- Use the property below to point to a specific JGroups configuration file on your classpath -->
                   <properties>
                    <property name="configurationFile" value="jgroups-tcp.xml" />
                   </properties>
                  </transport>
                </global>
                <default>
                  <storeAsBinary enabled="true"/>
                  <locking
                   isolationLevel="READ_COMMITTED"
                   lockAcquisitionTimeout="20000"
                   writeSkewCheck="false"
                   concurrencyLevel="5000"
                   useLockStriping="false"
                  />
                  <jmxStatistics enabled="true"/>
                 
                <clustering mode="replication">
                   <!-- Defines whether to retrieve state on startup.  This is only valued for 'replication' mode. -->
                   <stateTransfer
                    timeout="240000"
                    fetchInMemoryState="false" />

                   <!-- Network calls are synchronous. -->
                   <sync replTimeout="20000"/>
                </clustering>
                </default>
                <!--  A cache where data can be evicted if not used after some time, or if it exceeds certain capacity.
                   As shown in this example, passivation can be configured so that whenever data is removed due
                   eviction/expiration, this is in turn stored in a back-end cache store. Afterwards, when data is
                   requested, the cache store will be checked and data will be activated and brought back to memory
                -->
                <!-- LRU: least-recently-used pattern -->
                <namedCache name="dogCache">
                  <eviction
                   maxEntries="5"
                   strategy="LRU"
                  />
                  <expiration
                   wakeUpInterval="500"
                   lifespan="60000"
                   maxIdle="1000"
                  />
                </namedCache>
                <namedCache name="pets">
                  <eviction
                   maxEntries="10"
                   strategy="LRU"
                  />
                  <expiration
                   wakeUpInterval="500"
                   lifespan="60000"
                   maxIdle="1000"
                  />
                </namedCache>
                <namedCache name="catCache">
                  <eviction
                   maxEntries="5"
                   strategy="LRU"
                  />
                  <expiration
                   wakeUpInterval="500"
                   lifespan="60000"
                   maxIdle="1000"
                  />
                </namedCache>
                </infinispan>

                Thanks!

                • 5. Re: Updating cache on other nodes
                  Tristan Tarrant Master

                  I think I see where the problem is:

                   

                  cache configurations do not inherit from one another, therefore the namedcaches you have there will be simple local caches. The default cache (cacheManager.getCache()) and any non-defined named-cache will instead be clustered. My suggestion is to add the <clustering> section to all named caches

                  1 of 1 people found this helpful
                  • 6. Re: Updating cache on other nodes
                    sannsims Newbie

                    Per your suggestion, I modified the configuration and added <clustering> to all named caches.  I removed the other two caches and now only have one named cached with a clustering element (see below).  Unfortunately, this did not seem to make a difference.  I am still seeing a problem with caching, sequential requests still goes to the data broker - the cache is still empty.  Any other suggestions, please?

                     

                    <namedCache name="pets">
                      <eviction
                       maxEntries="10"
                       strategy="LRU"
                      />
                      <expiration
                       wakeUpInterval="500"
                       lifespan="60000"
                       maxIdle="1000"
                      />
                      <clustering mode="replication">
                       <!-- Defines whether to retrieve state on startup.  This is only valued for 'replication' mode. -->
                       <stateTransfer
                        timeout="240000"
                        fetchInMemoryState="false" />

                       <!-- Network calls are synchronous. -->
                       <sync replTimeout="20000"/>

                       <!-- Uncomment this for async replication.
                       <async
                        useReplQueue="true"
                                 replQueueInterval="10000"
                                 replQueueMaxElements="500" />
                       -->
                      </clustering>
                    </namedCache>
                    • 7. Re: Updating cache on other nodes
                      sannsims Newbie

                      Thanks Tristan, updating the named cache with a clustering element partially fixed my issue.  I discovered I also needed to "start" the cache, by starting the cache, I am now seeing the cache maintained on the first server.

                       

                      Thanks again!

                      • 8. Re: Updating cache on other nodes
                        sannsims Newbie

                        Tristan, I spoke too soon!  I thought this issue was fixed when "start()" was added, however it's still doesn't work.  I'm still seeing problems with the cache disappearing on subsequent calls. 

                         

                        This is what the logs show:

                         

                        21:35:55,943 INFO [ResteasyDeployment] Adding singleton resource com.infinispan.rest.services.PetService from Application javax.ws.rs.core.Application

                        21:35:56,053 INFO [STDOUT] ***** 1. IS PET CACHE EMPTY: true *****

                        21:35:56,069 INFO [STDOUT] Pet Cache keySet size: 0

                        21:35:56,069 INFO [JGroupsTransport] ISPN000078: Starting JGroups Channel

                        21:35:56,256 INFO [STDOUT]

                        -------------------------------------------------------------------

                        GMS: address=Node-A-46907, cluster=infinispan-cluster, physical address=127.0.0.1:7802

                        -------------------------------------------------------------------

                        21:35:56,272 INFO [JGroupsTransport] ISPN000094: Received new cluster view: [Node-A-47491|2] [Node-A-47491, Node-A-24936, Node-A-46907]

                        21:35:56,850 INFO [JGroupsTransport] ISPN000094: Received new cluster view: [Node-A-47491|2] [Node-A-47491, Node-A-24936, Node-A-46907]

                        21:35:56,881 INFO [JGroupsTransport] ISPN000094: Received new cluster view: [Node-A-47491|2] [Node-A-47491, Node-A-24936, Node-A-46907]

                        21:35:56,897 INFO [JGroupsTransport] ISPN000079: Cache local address is Node-A-46907, physical addresses are [127.0.0.1:7802]

                        21:35:56,928 INFO [CacheJmxRegistration] ISPN000031: MBeans were successfully registered to the platform mbean server.

                        21:35:56,991 INFO [STDOUT] Executing PetDAO getAllPets()

                        21:35:57,585 INFO [STDOUT] 2. Is pet cache empty now: false

                        21:35:57,585 INFO [STDOUT] Pet Cache keySet size: 1

                        21:35:57,585 INFO [STDOUT] Executing displayMessage on the Rest Server!

                        21:36:11,226 INFO [STDOUT] ***** 1. IS PET CACHE EMPTY: true *****

                        21:36:11,226 INFO [STDOUT] Pet Cache keySet size: 0

                        21:36:11,226 INFO [STDOUT] Executing PetDAO getAllPets()

                        21:36:11,289 INFO [STDOUT] 2. Is pet cache empty now: false

                        21:36:11,289 INFO [STDOUT] Pet Cache keySet size: 1

                        21:36:11,289 INFO [STDOUT] Executing displayMessage on the Rest Server!

                        21:36:17,243 INFO [STDOUT] ***** 1. IS PET CACHE EMPTY: true *****

                        21:36:17,243 INFO [STDOUT] Pet Cache keySet size: 0

                        21:36:17,243 INFO [STDOUT] Executing PetDAO getAllPets()

                        21:36:17,290 INFO [STDOUT] 2. Is pet cache empty now: false

                        21:36:17,290 INFO [STDOUT] Pet Cache keySet size: 1

                        21:36:17,306 INFO [STDOUT] Executing displayMessage on the Rest Server!

                         

                        Here you can see, each repeated attempt to get the "pet cache", it initially is empty until after I call the broker.  No change to the configuration file, the "pets" namedCache has a clustered element defined as replication.  I've slightly updated the code however, so here it is again:

                         

                        public DemoCacheManager() throws Exception {
                          System.out.println( "Executing DemoCacheManager constructor" );
                          mgr = new DefaultCacheManager( "infinispan.xml" );

                          petCache = mgr.getCache( "pets" );
                          petCache.start();
                        }

                        public List<Pet> buildPetCache() {
                          ArrayList<Pet> petList = new ArrayList<Pet>();
                          try {
                           System.out.println( " ***** 1. IS PET CACHE EMPTY: " + petCache.isEmpty() + " *****" );
                           System.out.println( "Pet Cache keySet size: " + petCache.keySet().size() );
                           if( mgr.getCache( "pets" ).isEmpty() ) {
                            petList = (ArrayList<Pet>) dao.getAllPets();  //getting data from database
                            System.out.println( "2. Is pet cache empty now: " + petCache.isEmpty() );
                            System.out.println( "Pet Cache keySet size: " + petCache.keySet().size() );   
                            petCache.put( new Integer(0), petList ); 
                           }
                          }
                          catch( Exception ex ) {
                           ex.printStackTrace();
                          }
                          return petList;
                        }

                         

                         

                        I've looked at sample code and I don't seem to be doing anything different?  Any suggestions is greatly appreciated!

                        Thanks for your help!