10 Replies Latest reply on Mar 5, 2013 7:12 AM by rhusar

    Heavy multicast traffic created by Jboss 5 AS cluster

    maali

      Hi,

       

      We have a Jboss 5 AS cluster consiteing of 2 nodes using multicast, every thing works fine and the servers are able to discover and make a cluster

       

      but the problem is these servers generate heavy multicast traffic which effects the network performace of other servers shareing the same network.

       

      I am new to Jboss clustering is there any way to use unicast (point-to-point) instead of multicast ? Or configure the multicast such that its not problem for rest of the network ? can you refer me to some documentation , blog post or simmillar that can help me get rid of this problem.

       

      Thanks for the help.

        • 1. Re: Heavy multicast traffic created by Jboss 5 AS cluster
          rhusar

          Hey Maali,

           

          If you have 2 nodes, there is not a great benefit of using multicast so you can switch to unicast.

           

          You can do this by switching to TCP steak instead. You can use this parameter on startup:

           

          -Djboss.default.jgroups.stack=tcp

          Take a look at this configuration file where you can configure that:

          server/production/deploy/cluster/jboss-cache-manager.sar/META-INF/jboss-cache-manager-jboss-beans.xml

          futhermore look at the JGroups config located nearby.

           

           

          Regarding not flooding the network, you can (and you should) configure number of hops to 1 by chanding the TTL value.

           

          HTH,

          Rado

          1 of 1 people found this helpful
          • 2. Re: Heavy multicast traffic created by Jboss 5 AS cluster
            maali

            Hi Radoslav,

             

            Thanks for your reply, Currently its 2 nodes but there exist a posibility that more nodes are added in future and multicast offers the flexibility where nodes can discover the new ones that are added.

             

            You mentioned the TTL property would something like this work in startup script "-Djgroups.udp.ip_ttl=1" ? and can we keep on using the multicast and by setting the TTL cluster will generate less traffic ?

             

            BR

            1 of 1 people found this helpful
            • 3. Re: Heavy multicast traffic created by Jboss 5 AS cluster
              rhusar

              Hey

               

              The volume of data between the nodes will be the same of course, but depedending on your topology, limiting the TTL can keep the traffic from spreading further from the lab.

               

              What is the traffic mostly? HTTP Session? Maybe you should look at what you are actually replicating in the sessions andwhether its necessary, or if its becoming a problem tak a look at granular replication.

               

              Rado

              • 4. Re: Heavy multicast traffic created by Jboss 5 AS cluster
                maali

                Hi Rado,

                 

                Yes the traffice is mainly HTTP sessions, we will try the TTL solution and see if that works.

                Will update the thread.

                • 5. Re: Heavy multicast traffic created by Jboss 5 AS cluster
                  maali

                  Hi Rado,

                   

                  a bit late update but actually the TTL change fixed it so didnt had to change any other settings just set it in the clustering options in startup script.

                   

                  Thanks a lot for the help.

                  • 6. Re: Heavy multicast traffic created by Jboss 5 AS cluster
                    rhusar

                    Fantastic, thanks for the update.

                     

                    (I will document this "Clustering troubleshooting" document for others.)

                    • 7. Re: Heavy multicast traffic created by Jboss 5 AS cluster
                      maali

                      Hi Radoslav,

                       

                      Thanks for you great support, as mentioned we managed to fix it in one of our enviornments.

                       

                      Now we are having another enviornment which has a diffrenct network atchitecture then the one where TTL fix worked. In this enviornment if we change the TTL to 1 the

                       

                      servers don't discover each other at all and cluster is not formed.

                       

                      As changing the architecture is not an option so I was looking into your idea about unicast clustering , can you elaborate it further and refer me to some documentation links for Jboss 5.

                       

                      BR

                      • 8. Re: Heavy multicast traffic created by Jboss 5 AS cluster
                        rhusar

                        Hi Maali,

                         

                        So lets look into unicast clustering instead of UDP mcast. Open up your profile and look into file jgroups-channelfactory-stacks.xml and locate the stack named "tcp". That stacks still uses UDP only for multicast discovery. If low UDP traffic is alright, you dont need to change it. If it is or mcast doesn't work, you will need to configure TCPPING protocol and configure intial_hosts where to look for cluster members.

                         

                        Afterwards, you will need to tell JBoss Cache to use this stack, open up jboss-cache-manager-jboss-beans.xml where for each cache you have a stack defined. You can either change it here from udp to tcp or you can simply use the property when starting AS, just add:

                         

                        -Djboss.default.jgroups.stack=tcp

                         

                        HTH,

                        Rado

                        • 9. Re: Heavy multicast traffic created by Jboss 5 AS cluster
                          maali

                          Hi Radoslav,

                           

                          Thanks for your quick response, I have implemented the solution and it works in our developement env will have to wait and see how it goes in the target enviornment when

                          its updated.

                           

                          BR

                          • 10. Re: Heavy multicast traffic created by Jboss 5 AS cluster
                            rhusar

                            I always stress out, that before going to production to stress test your application or at least stress test the network using some sort of clustering benchmark. TCP scales worse than UDP for larger clusters (say 4 or larger).