4 Replies Latest reply on Aug 5, 2010 9:56 AM by shane_dev

    Hot Rod Clarifications

    shane_dev

      Assume that we have a Hot Rod cluster (A and B) set up for replication, and that we have to clients (1 and 2)

       

      When using a remote cache store on the clients, can 'fetchPersistentState' be used to have the clients retrieve all entries in the Hot Rod cluster at start up?

       

      If not, I also noticed a 'loadAll' method. Can this be used to force the clients to retrieve all entries in the Hot Rod cluster at startup?

       

      I noticed that ISPN-374 is still open.

      • Does that mean we run the risk of having stale entries on our clients?
      • Also, even if we did load all of the entries at start up, does that mean that if subsequent entries were added via 1 that they would not end up on 2 unless it specifically requested them?
      • It looks like ISPN-374 is geared towards invalidation. I'm wondering if it can or will be expanded to actually push the 'entries' instead of just notifications based on the key.

       

      From what I can tell, it looks like the remote cache store is intended to operate as an L1 cache. We were considering the use of Hot Rod as a means to survive application cluster failures by separating the cache. However, now it looks like we just want a 4 node cluster such that 2 nodes are on the same VMs as our application while 2 nodes are just running Infinispan. Thus we could lose our applications, but the cache would survive.

       

      On the otherhand, as an L1 cache, we may not be able to handle the delay in fetching the entry over the wire each time. That's why I'm wondering if I can have the best of both worlds by having the Hot Rod instances replicate with the clients.

        • 1. Re: Hot Rod Clarifications
          galder.zamarreno

          As part of CR2, https://jira.jboss.org/browse/ISPN-516 and https://jira.jboss.org/browse/ISPN-539 were implemented which enabled both preloading and  fetchPersistentState to be used with RemoteCacheStore. Your code should not be calling to the CacheStore API.

           

          If your Hot Rod client is a part of remote cache store, then it means that any local modifications will be transfered to the remote Hot Rod servers. You only risk stale entries if the Infinispan instances on the client side are not clustered cos they won't know about each others modifications. IOW, you need your Infinispan instances in 1 and 2 clients to be clustered to avoid stale reads. These two better be in a different cluster to the Hot Rod cluster to avoid mixing up traffic.

           

          ISPN-374 (https://jira.jboss.org/browse/ISPN-374) plans to include notifications on a per key basis. The forum thread linked from the JIRA explains it better.

           

          The remote cache store is intended as a more persistent store where data can survive to crashes in your main nodes. In a way, it can act as a L1 with only data recently used (having eviction configured). If data cannot be found in the main Infinispan instances, it can be retrieved from the backed remote cache store.

           

          Generally, you keep the remote cache store nodes in separate machines to the client side Infinispan instances, so that data can survive node failures.

          • 2. Re: Hot Rod Clarifications
            shane_dev

            Can you clarify if fetchInMemoryState is now supported?

             

            I was looking at the documentation and it seems mixed.

             

            http://docs.jboss.org/infinispan/4.1/apidocs/org/infinispan/loaders/remote/RemoteCacheStore.html

             

            "Due to certain HotRod constraints, this cache store does not support preload and also cannot be used for provide state. Setting fetchPersistentState is not allowed."

             

            https://jira.jboss.org/browse/ISPN-516

             

            "This would not load not the entire state, but up to N entries."

             

            http://community.jboss.org/wiki/HotRodBulkGet-Design

             

            "Following method won't be supported by RemoteCacheStore (see Manik's first comment for more details on why this is not supported): public Set<Object> loadAllKeys(Set<Object> keysToExclude)"

             

            "entry count [vint] : maximum number of Infinispan entries to be returned by the server (entry == key + associated value). Needed to support CacheLoader.load(int). If 0 then all entries are returned (needed for CacheLoader.loadAll())."

             

            "The same logic and code from state generation, in case of in memory state retrieval can be used <stateRetrieval fetchInMemoryState="true"/>"

             

            "No locking is performed: this is fast but consistency might suffer."

             

            Jira Issues: 516 & 539 (Hot Rod / RemoteCacheStore - State Transfer)

             

            Both are resolved.

             

             

            It seems like there are 3 different views on this matter:

             

            1. State Transfer is NOT Supported.
            2. State Transfer is LIMITED to N Entries.
            3. State Transfer is FULLY Supported.
            • 3. Re: Hot Rod Clarifications
              galder.zamarreno

              It looks to me that the RemoteCacheStore javadoc has not been updated. Back in CR1 (http://anonsvn.jboss.org/repos/infinispan/tags/4.1.0.CR1/cachestore/remote/src/main/java/org/infinispan/loaders/remote/RemoteCacheStore.java), toStream/fromStream were not implemented and hence at that point fetching persistent state was not supported. However, in CR2, this was resolved (http://anonsvn.jboss.org/repos/infinispan/tags/4.1.0.CR2/cachestore/remote/src/main/java/org/infinispan/loaders/remote/RemoteCacheStore.java) and hence this should be updated.

               

              With bulk gets Hot Rod can retrieve all, or a number of, elements.

              • 4. Re: Hot Rod Clarifications
                shane_dev

                Thanks Galder,

                 

                I had assumed that was the case, but after looking at the documentation I wanted to confirm before we headed down the Hot Rod course.

                 

                One thing that caught my interest was the quote on inconsistency.

                 

                "No locking is performed: this is fast but consistency might suffer."

                 

                It got me to wondering, is this at all related to the state transfer / repl queue issue we have been discussing? I realize this is from the Hot Rod article but I figure it applies to non block state transfer in general. Or, perhaps it is specific to the Hot Rod state transfer.