13 Replies Latest reply on Jun 1, 2006 4:18 AM by Bela Ban

    Yed clustering setup

    David Webb Newbie

      I have singe physical machine that one NIC and I need to setup 3 clusters on the machine for 3 different applications.

      My setup is:
      Linux
      JBoss 4.0.1SP1
      JDK 1.4.2_03
      Tomcat 5

      My experience with clustering J2EE servers tells me that changing the mcast_addr in the cluster_services.xml and tc5-clustering-service.xml files to be unique to each cluster will separate the clusters and the clustering activity. My example will only use 2 servers since I can not even separate 2 clusters at this point.

      Server 1 - Cluster 1

      cluster-service.xml

      <UDP mcast_addr="228.1.2.41" mcast_port="45566" bind_addr="192.168.1.41"
       ip_ttl="32" ip_mcast="true"
       mcast_send_buf_size="800000" mcast_recv_buf_size="150000"
       ucast_send_buf_size="800000" ucast_recv_buf_size="150000"
       loopback="false" />
      


      tc5-cluster-service.xml

      <UDP mcast_addr="230.1.2.41" mcast_port="45577" bind_addr="192.168.1.41"
       ip_ttl="64" ip_mcast="true"
       mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
       ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
       loopback="false"/>
      


      Server 2 - Cluster 2

      cluster-service.xml

      <UDP mcast_addr="228.1.2.42" mcast_port="45566" bind_addr="192.168.1.42"
       ip_ttl="32" ip_mcast="true"
       mcast_send_buf_size="800000" mcast_recv_buf_size="150000"
       ucast_send_buf_size="800000" ucast_recv_buf_size="150000"
       loopback="false" />
      


      tc5-cluster-service.xml

      <UDP mcast_addr="230.1.2.42" mcast_port="45577" bind_addr="192.168.1.42"
       ip_ttl="64" ip_mcast="true"
       mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
       ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
       loopback="false"/>
      


      The results are as follows. The first server comes up no problem and deploys the expected packages in the deploy and farm directories. When I bring up the second server, it joins a cluster with server 1 and picks up all the files in server 1's farm directory.

      What else do I need to change to have 2 to n clusters on a single machine when each cluster is bound to a different IP address?

      Thank you in advance for any assistance.

        • 1. Re: multihomed clustering setup
          David Webb Newbie

          I dont know how the thread subject made it to "Yed Clustering Setup", but it was meant to be "Multihomed Clustering Setup". If an admin could modify the subject, perhaps it will be more helpful.

          • 2. Re: Yed clustering setup
            David Webb Newbie

            I wanted to add that leaving the mcast_addr the same for all servers and changing the mcast_port does effectively separate the clusters on the same machine. However, I think the mcast_addr change by itself should work and I welcome any explanation or help.

            Thank you.

            • 3. Re: Yed clustering setup
              David Webb Newbie

              I actually have 2 NICs on the machine and splitting the servers between the 2 physical IP addresses with different multicast addresses works properly.

              My delimma is that I have only 2 NICs and I need 3 clusters on the linux box. I have created an additional IP address for one of the NICs using an IP alias.


              eth0 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
               inet addr:192.168.1.11 Bcast:192.168.1.255 Mask:255.255.255.0
               inet6 addr: fe80::20f:1fff:fe66:7441/64 Scope:Link
               UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
               RX packets:339235 errors:0 dropped:0 overruns:0 frame:0
               TX packets:288828 errors:0 dropped:0 overruns:0 carrier:0
               collisions:0 txqueuelen:100
               RX bytes:43175205 (41.1 Mb) TX bytes:233529905 (222.7 Mb)
               Interrupt:16 Memory:fcf30000-fcf40000
              
              eth0:0 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
               inet addr:192.168.1.40 Bcast:192.168.1.255 Mask:255.255.255.0
               UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
               Interrupt:16 Memory:fcf30000-fcf40000
              
              eth0:1 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
               inet addr:192.168.1.41 Bcast:192.168.1.255 Mask:255.255.255.0
               UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
               Interrupt:16 Memory:fcf30000-fcf40000
              
              eth0:2 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
               inet addr:192.168.1.42 Bcast:192.168.1.255 Mask:255.255.255.0
               UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
               Interrupt:16 Memory:fcf30000-fcf40000
              
              eth0:3 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
               inet addr:192.168.1.43 Bcast:192.168.1.255 Mask:255.255.255.0
               UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
               Interrupt:16 Memory:fcf30000-fcf40000
              
              eth1 Link encap:Ethernet HWaddr 00:0F:1F:66:74:42
               inet addr:192.168.1.12 Bcast:192.168.1.255 Mask:255.255.255.0
               inet6 addr: fe80::20f:1fff:fe66:7442/64 Scope:Link
               UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
               RX packets:58315 errors:0 dropped:0 overruns:0 frame:0
               TX packets:138 errors:0 dropped:0 overruns:0 carrier:0
               collisions:0 txqueuelen:100
               RX bytes:15660151 (14.9 Mb) TX bytes:8880 (8.6 Kb)
               Interrupt:17 Memory:fcf10000-fcf20000

              While everything else seems to bind properly to the IP alias on the NIC, the clustering portion of JBoss clusters all server on the physical NIC, regardless of the IP address its bound to.

              Has anyone sucessfully implemented more than one cluster on the same NIC using an IP alias and what must I change in addition to the mcast_addr and bind_addr in the cluster-service.xml and tc5-cluster-service.xml files?

              Thanks in advance.

              • 4. Re: Yed clustering setup
                Ben Wang Master

                Not usre if the subject can be changed at this stage.

                Anyway, your problem is related to how do I start multiple JBoss instance in a single machine. Couple different ways: different ips or different ports.

                Check the wiki. It should have the details.

                -Ben

                • 5. Re: Yed clustering setup
                  David Webb Newbie

                  I have different IPs, the problem seems to be that the different IPs are all bound to the same physical NIC via Linux Virtual IPs. Any suggestions? I have read all the Wiki's. I understand how to set it up, but it is not behaving properly as described above.

                  • 6. Re: Yed clustering setup
                    Ben Wang Master

                    If that were the case, is the 1 ip with 3 different set of ports not an option?

                    -Ben

                    • 7. Re: Yed clustering setup
                      David Webb Newbie

                      Of course it is an option and the way I am curretnly implementing it. Im asking so you can better your product and have it perform like your competition does. Is it a bug? Do I need to report it to JGroups or does JGroups monitor this forum?

                      Do you see my point?

                      • 8. Re: Yed clustering setup
                        Jay Glass Novice

                        I am also having issues...

                        I am not neccessarily clustering though, but it is related, and was wondering if anyone could help...

                        I have two instances of JBoss Application Server, at the moment, only one is set to run, the JBAS is actually running a JBoss Mail server instance

                        I have 1 NIC, with ten IP addresses assigned to it.

                        Even though I start up JBoss application server using run.bat -b 216.122.145.50

                        and it is listening on the correct IP(216.122.145.50) and ports, when it connects to an outside server, i.e. on the internet it actually connects using the ip address of 216.122.145.49

                        this is on a windows box

                        in the routing table, there is only one interface listed, as stated, I am only using one interface, though there are 10 ip's bound to it

                        the one interface, shows it's IP address as 216.122.145.49

                        is routing causing the issue? or does jboss boss server, listen on whatever port and IP you tell it too, but communicate to other servers grabbing whatever IP address it can?

                        any help, and insight would be greatly appreciated

                        • 9. Re: MultiHomed clustering setup
                          David Webb Newbie

                          Based on your posting, I do not think that is the same issue I was having with the MultiCast Address used in clustering.

                          • 10. Re: Yed clustering setup
                            Jay Glass Novice

                            thanks davewebb, this is so frustrating...

                            • 11. Re: Yed clustering setup
                              Brian Stansberry Master

                              David,

                              Perhaps http://wiki.jboss.org/wiki/Wiki.jsp?page=PromiscuousTraffic is the explanation for your cluster separation issue.

                              gohip,

                              Hijacking a thread with a question that is off-topic to the forum is normally not the best way to get an answer. But... in general the -b switch only controls the interface used for sockets JBoss opens to listen on.

                              • 12. Re: Yed clustering setup
                                David Webb Newbie

                                That is my problem exactly. Thanks for the closure.

                                • 13. Re: Yed clustering setup
                                  Bela Ban Master

                                  This (http://wiki.jboss.org/wiki/Wiki.jsp?page=LoopbackAdaptor) may be the same problem. You should try setting null_src_addresses="true" in your 2.2.8 XML config file. Note that this flag was removed in 2.2.9, so it should work there. The associated JIRA issue was for Windows, so I don't know if the same applies to your problem...