7 Replies Latest reply on May 13, 2010 7:15 AM by galder.zamarreno

    Multicast discovery with separate clusters

    cbo_

      I would like to allow discover to occur in my environment via multicast, but also would like to set up separate clusters.  Once discovery is complete I would like the distribution of caches to occur within each cluster to occur without interference from other clusters.  I was thinking I might be able to get away with setting separate clusterName=  entries in transport element of my global config.  However, I am seeing a lot of discard messages in the logs of my apps.  Currently, I am not specifying any other properties within the transport element.  Wonder if there are any docs out there or quick suggestions some of you can make.

       

      Some of the messages I am seeing that I would like to eliminate:

      2010-05-04 11:47:15,082 [OOB-13,test9-23195] WARN  org.jgroups.protocols.UDP - discarded message from different cluster "infinispan-cluster-test16" (our cluster is "infinispan-cluster-test9"). Sender was 73ff33fe-180d-6170-9c34-e6a9c1041ff7

        • 1. Re: Multicast discovery with separate clusters
          manik

          Your best bet really is to use different multicast addresses and ports.  See PromiscuousTraffic for more details.

          • 2. Re: Multicast discovery with separate clusters
            cbo_

            I have now set up a separate cluster between different development environments.  For the sake of discussion, let's call one environment devA and the other devB.  I will have 2 machines in each of the 2 environments.  It seems I have the multicast set up to allow the separation I thought I wanted.  However, now I have discovered my next item of interest.  I have 2 JVMs on each of the machines (devA has machine1 with 2 JVMs and machine 2 with 2 JVMs....same for devB).  Now, just focusing on one environment, devA, I would like the JVMa on machine1 and JVMa on machine 2 to share the same replicated named cache (sharedCache_a).  And, I would like JVMb on machine1 and JVMb on machine2 to share their same replicated named cache (sharedCache_b).  Also, I am setting the StateRetrieval fetchInMemoryState=true for both named caches.  I was seeing state being replicated on startup when I was running only the JVMa processes on both machines.  Also, if only starting JVMb processes on both machines I was also seeing state being replicated on startup.  However, now that I am running JVMa and JVMb on both machines I am seeing state replicated between JVMa processes, but I am not seeing the state replicated between JVMb processes.  One thing I noticed, after digging through detail logs, is that the coordinator of the caches seems to be associated with JVMa on machine1.  Since it is not connected to sharedCache_b it does not have state to replicate on startup to the JVMb processes.  Interesting that after caches are started, replication does work as expected, but the "catch-up" of state on startup is not working as described above.

             

            Do I need to further separate the cluster to only include like-JVMs (a or b i this example)?  Ultimately what I thought I would get is cache coordinators from each cache name.  Seems it is per cluster.

            • 3. Re: Multicast discovery with separate clusters
              manik

              Do the different JVMs form different clusters?  E.g., does devA = {machine1_JVMa, machine2_JVMa} and devB = {machine1_JVMb, machine2_JVMb} ?

              • 4. Re: Multicast discovery with separate clusters
                cbo_

                In the way my application considers a cluster, the answer is no.  A cluster in this case would be made up of machine1_JVMa, machine1_JVMb, machine2_JVMa, and machine2_JVMb.

                • 5. Re: Multicast discovery with separate clusters
                  cbo_

                  Or, another way to put it, all JVMs on machine1 and machine2 make up the cluster.  And, in case it is helpful, I mentioned the cache mode is replication and the scenario I am attempting to use this is for high availability.

                  • 6. Re: Multicast discovery with separate clusters
                    cbo_

                    Having some coffee in the system now....re-read your response and realize I still did not answer your question.  Sorry.  In development I would have around 20 different clusters with a 2 machines in each cluster and a total of 10 JVMs in each cluster.  In production, the number of clusters would be greater, but the number of JVMs per cluster and the number of machines per cluster would be the same.

                    • 7. Re: Multicast discovery with separate clusters
                      galder.zamarreno

                      Craig, it's still unclear to me what your problem is. Separating each cluster is a matter of making sure that each cluster uses a different multicast address, multicast port and cluster name. The first two are configured in the JGroups protocol stack XML (mcas_port and mcast_addr properties of UDP element) or via system properties (i.e. http://community.jboss.org/wiki/SystemProps). The cluster name is defined in Infinispan's transport clusterName property (see http://docs.jboss.org/infinispan/4.1/apidocs/config.html#ce_global_transport).

                       

                      If this is working correctly, iow, if u can see both clusters properly isolated (grep by 'view' in the logs to see what each node considers to be the cluster view), the replication and state transfer should work as expected.

                       

                      If this is not working, you're gonna have to be more precise, show configurations used, properties in place, log lines that might be suspicious...etc.