4 Replies Latest reply on Jul 18, 2014 8:42 AM by thammoud

    Is Cross-Site invalidation supported?

    thammoud

      Hello,

       

      We are using JBOSS 8.1. We have two data centers. We would like to setup a cluster in each. Any cached data that is updated in one site needs to be invalidated in the other site. Everything works if we setup all machines in all data centers to be in one cluster. That will force us to use TCP for all invalidation requests which is not the most efficient way of doing it. We thought that a better setup will be to use multicast in each data center and take advantage of RELAY2 which we setup in the our standalone.xml. This will allow for only one TCP message to be sent cross-site where multicast will be used on the other end.

       

      We tested against two nodes each in a different data center. Each node holds a cache and is the coordinator in their respective cluster. That does not seem to work. Looking at the JMS console for JGroups, I can see the "hibernate" channel in each server with one self entry as to be expected as the clusters are independent. There is another "hibernate/global" channel created as a result of the relay setup. That view contains both machines.

       

      Looking closely at the code and specifically at JGroupsTransport.java around line 516 (invokeRemotely method), the logic seems to skip RPC as after you remove self from the list of addresses, there are no recipients. It's like the code does not expect xsite configuration. Our design approach seems to be flawed or not supported and any input to other alternatives will be greatly appreciated.

       

      Thank you for all your help

        • 1. Re: Is Cross-Site invalidation supported?
          rvansa

          JGroupsTransport does not handle the XSite, this is processed by org.infinispan.interceptors.xsite.*BackupInterceptor - JGroupsTransport.backupRemotely is later called to send the command to the other site.

           

          Was there any problem replicating the commands, or are you just exploring the sources? Set logging for TRACE level on org.jgroups and see whether the message is transported.

          • 2. Re: Is Cross-Site invalidation supported?
            thammoud

            Radim,

             

            Thanks for the quick response. The commands are not replicating. I am using an invalidation cache for the entities. It seems that a BackupSenderImpl is only started when there is a backup configuration. Backups (According to the infinispan schema) in Wildfly 8.1 can not be set on invalidation caches and only for shared state caches. None of the backup calls are being invoked.

            • 3. Re: Is Cross-Site invalidation supported?
              rvansa

              Ok, so if it cannot be configured, the fact that it's not working is rather a feature than a bug ;-) Invalidation caches don't get that much attention from developers as distributed/replicated caches.

              • 4. Re: Is Cross-Site invalidation supported?
                thammoud

                I have a feeling that there might be a disconnect between the Wildfly-8.1 infinispan schema and the native infinispan schema. In the native one, I looked at IS 7  schema (Which is NOT being used in JBOSS 8.x) and noticed that all the cache types can have the backups tag specified which means it should work the same for invalidation. I also looked at the 6.x schema (7 was heavily modified and actually is more readable) and still do not find such a limitation in infinispan. I believe that the author or the JBOSS 8.x infinispan schema knows something that I don't and chose not to base the invalidation cache the same way. I will need to find the author of the JBOSS xsd version and ask him for an explanation. Thank you for your help.