Yed clustering setup

[davewebb]

I have singe physical machine that one NIC and I need to setup 3 clusters on the machine for 3 different applications.

My setup is:
Linux
JBoss 4.0.1SP1
JDK 1.4.2_03
Tomcat 5

My experience with clustering J2EE servers tells me that changing the mcast_addr in the cluster_services.xml and tc5-clustering-service.xml files to be unique to each cluster will separate the clusters and the clustering activity. My example will only use 2 servers since I can not even separate 2 clusters at this point.

Server 1 - Cluster 1

cluster-service.xml

<UDP mcast_addr="228.1.2.41" mcast_port="45566" bind_addr="192.168.1.41"
 ip_ttl="32" ip_mcast="true"
 mcast_send_buf_size="800000" mcast_recv_buf_size="150000"
 ucast_send_buf_size="800000" ucast_recv_buf_size="150000"
 loopback="false" />

tc5-cluster-service.xml

<UDP mcast_addr="230.1.2.41" mcast_port="45577" bind_addr="192.168.1.41"
 ip_ttl="64" ip_mcast="true"
 mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
 ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
 loopback="false"/>

Server 2 - Cluster 2

cluster-service.xml

<UDP mcast_addr="228.1.2.42" mcast_port="45566" bind_addr="192.168.1.42"
 ip_ttl="32" ip_mcast="true"
 mcast_send_buf_size="800000" mcast_recv_buf_size="150000"
 ucast_send_buf_size="800000" ucast_recv_buf_size="150000"
 loopback="false" />

tc5-cluster-service.xml

<UDP mcast_addr="230.1.2.42" mcast_port="45577" bind_addr="192.168.1.42"
 ip_ttl="64" ip_mcast="true"
 mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
 ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
 loopback="false"/>

The results are as follows. The first server comes up no problem and deploys the expected packages in the deploy and farm directories. When I bring up the second server, it joins a cluster with server 1 and picks up all the files in server 1's farm directory.

What else do I need to change to have 2 to n clusters on a single machine when each cluster is bound to a different IP address?

Thank you in advance for any assistance.

[davewebb]

I dont know how the thread subject made it to "Yed Clustering Setup", but it was meant to be "Multihomed Clustering Setup". If an admin could modify the subject, perhaps it will be more helpful.

[davewebb]

I wanted to add that leaving the mcast_addr the same for all servers and changing the mcast_port does effectively separate the clusters on the same machine. However, I think the mcast_addr change by itself should work and I welcome any explanation or help.

Thank you.

[davewebb]

I actually have 2 NICs on the machine and splitting the servers between the 2 physical IP addresses with different multicast addresses works properly.

My delimma is that I have only 2 NICs and I need 3 clusters on the linux box. I have created an additional IP address for one of the NICs using an IP alias.

eth0 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
 inet addr:192.168.1.11 Bcast:192.168.1.255 Mask:255.255.255.0
 inet6 addr: fe80::20f:1fff:fe66:7441/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:339235 errors:0 dropped:0 overruns:0 frame:0
 TX packets:288828 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:100
 RX bytes:43175205 (41.1 Mb) TX bytes:233529905 (222.7 Mb)
 Interrupt:16 Memory:fcf30000-fcf40000

eth0:0 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
 inet addr:192.168.1.40 Bcast:192.168.1.255 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 Interrupt:16 Memory:fcf30000-fcf40000

eth0:1 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
 inet addr:192.168.1.41 Bcast:192.168.1.255 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 Interrupt:16 Memory:fcf30000-fcf40000

eth0:2 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
 inet addr:192.168.1.42 Bcast:192.168.1.255 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 Interrupt:16 Memory:fcf30000-fcf40000

eth0:3 Link encap:Ethernet HWaddr 00:0F:1F:66:74:41
 inet addr:192.168.1.43 Bcast:192.168.1.255 Mask:255.255.255.0
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 Interrupt:16 Memory:fcf30000-fcf40000

eth1 Link encap:Ethernet HWaddr 00:0F:1F:66:74:42
 inet addr:192.168.1.12 Bcast:192.168.1.255 Mask:255.255.255.0
 inet6 addr: fe80::20f:1fff:fe66:7442/64 Scope:Link
 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
 RX packets:58315 errors:0 dropped:0 overruns:0 frame:0
 TX packets:138 errors:0 dropped:0 overruns:0 carrier:0
 collisions:0 txqueuelen:100
 RX bytes:15660151 (14.9 Mb) TX bytes:8880 (8.6 Kb)
 Interrupt:17 Memory:fcf10000-fcf20000

While everything else seems to bind properly to the IP alias on the NIC, the clustering portion of JBoss clusters all server on the physical NIC, regardless of the IP address its bound to.

Has anyone sucessfully implemented more than one cluster on the same NIC using an IP alias and what must I change in addition to the mcast_addr and bind_addr in the cluster-service.xml and tc5-cluster-service.xml files?

Thanks in advance.

[ben.wang]

Not usre if the subject can be changed at this stage.

Anyway, your problem is related to how do I start multiple JBoss instance in a single machine. Couple different ways: different ips or different ports.

Check the wiki. It should have the details.

-Ben

[davewebb]

I have different IPs, the problem seems to be that the different IPs are all bound to the same physical NIC via Linux Virtual IPs. Any suggestions? I have read all the Wiki's. I understand how to set it up, but it is not behaving properly as described above.

[ben.wang]

If that were the case, is the 1 ip with 3 different set of ports not an option?

-Ben

[davewebb]

Of course it is an option and the way I am curretnly implementing it. Im asking so you can better your product and have it perform like your competition does. Is it a bug? Do I need to report it to JGroups or does JGroups monitor this forum?

Do you see my point?

[gohip]

I am also having issues...

I am not neccessarily clustering though, but it is related, and was wondering if anyone could help...

I have two instances of JBoss Application Server, at the moment, only one is set to run, the JBAS is actually running a JBoss Mail server instance

I have 1 NIC, with ten IP addresses assigned to it.

Even though I start up JBoss application server using run.bat -b 216.122.145.50

and it is listening on the correct IP(216.122.145.50) and ports, when it connects to an outside server, i.e. on the internet it actually connects using the ip address of 216.122.145.49

this is on a windows box

in the routing table, there is only one interface listed, as stated, I am only using one interface, though there are 10 ip's bound to it

the one interface, shows it's IP address as 216.122.145.49

is routing causing the issue? or does jboss boss server, listen on whatever port and IP you tell it too, but communicate to other servers grabbing whatever IP address it can?

any help, and insight would be greatly appreciated

[davewebb]

Based on your posting, I do not think that is the same issue I was having with the MultiCast Address used in clustering.

[gohip]

thanks davewebb, this is so frustrating...

[brian.stansberry]

David,

Perhaps http://wiki.jboss.org/wiki/Wiki.jsp?page=PromiscuousTraffic is the explanation for your cluster separation issue.

gohip,

Hijacking a thread with a question that is off-topic to the forum is normally not the best way to get an answer. But... in general the -b switch only controls the interface used for sockets JBoss opens to listen on.

[davewebb]

That is my problem exactly. Thanks for the closure.

[belaban]

This (http://wiki.jboss.org/wiki/Wiki.jsp?page=LoopbackAdaptor) may be the same problem. You should try setting null_src_addresses="true" in your 2.2.8 XML config file. Note that this flag was removed in 2.2.9, so it should work there. The associated JIRA issue was for Windows, so I don't know if the same applies to your problem...