Jboss Clustering over TCP

[xav.vijay]

Hi

I am trying to create a jboss cluster of 2 nodes. Here is the setup required.

1. NodeA is on server1 on VLAN1
2. NodeB is on server2 on VLAN2.
3. No UDP Multicast traffic allowed.

What I have tried so far is ... in cluster-service.xml on both the servers

in server1

<TCP bind_addr="server1" start_port="7800" loopback="true"
tcp_nodelay="true"
recv_buf_size="20000000"
send_buf_size="640000"
discard_incompatible_packets="true"
enable_bundling="false"
max_bundle_size="64000"
max_bundle_timeout="30"
use_incoming_packet_handler="true"
use_outgoing_packet_handler="false"
down_thread="false" up_thread="false"
use_send_queues="false"
sock_conn_timeout="300"
skip_suspected_members="true"/>
<TCPPING initial_hosts="server2[7800]" port_range="3"
timeout="3000"
down_thread="false" up_thread="false"
num_initial_members="3"/>
<MERGE2 max_interval="100000"
down_thread="false" up_thread="false" min_interval="20000"/>
<FD_SOCK down_thread="false" up_thread="false"/>
<FD timeout="10000" max_tries="5" down_thread="false" up_thread="false" shun="true"/>
<VERIFY_SUSPECT timeout="1500" down_thread="false" up_thread="false"/>
<pbcast.NAKACK max_xmit_size="60000"
use_mcast_xmit="false" gc_lag="0"
retransmit_timeout="300,600,1200,2400,4800"
down_thread="false" up_thread="false"
discard_delivered_msgs="true"/>
<pbcast.STABLE stability_delay="1000" desired_avg_gossip="50000"
down_thread="false" up_thread="false"
max_bytes="400000"/>
<pbcast.GMS print_local_addr="true" join_timeout="3000"
down_thread="false" up_thread="false"
join_retry_timeout="2000" shun="true"
view_bundling="true"/>
<pbcast.STATE_TRANSFER down_thread="false" up_thread="false" use_flush="false"/>


and in server2, the bind_addr="server2" and TCPPING initial_hosts="server1[7800]" port_range="3".... everything else is same.

This didnt work... they didn't see each other. is there any other file which needs to be modified apart from cluster-service.xml???

More info on the servers.... I can ping server1 from server2 and vice versa
I am connecting to server2 from server1 via ssh and vice versa is also possible.

I tied the following config as well...instead of using the multicast over UDP, I tried multicast over TCP

<MPING timeout="2000"
bind_to_all_interfaces="true"
mcast_addr="224.0.0.1"
mcast_port="7500"
ip_ttl="8"
num_initial_members="3"
down_thread="false" up_thread="false"/>


The same config on both the servers. But this config gives me errors while starting.

More info.... 224.0.0.1 is the only multicast IP working in our environment. i can ping this IP from both server1 & server2. I dont know if I am missing something or doing something wrong.
Are there any other files which needs to be changed? is it possible to cluster 2 nodes across different VLAN's??

I would greatly appreciate any help.

Regards

Antonio

[brian.stansberry]

What AS release?

The cluster-service.xml isn't the only relevant config file. There are also JGroups channels opened for HttpSession replication, EJB3 SFSB replication and EJB3 entity caching (if your AS version supports EJB3). Where in deploy the configs for those are depends on what AS release.

See http://www.jboss.org/community/wiki/TestingJBoss for a procedure for trying to debug problems with nodes communicating with each other over JGroups.

[xav.vijay]

The Jboss release is 4.2.1 GA.

I have managed to get the cluster working between the 2 servers, by using their IP's instead of their hostnames.

I will make the necessary changes in the other files which you have mentioned to see how it goes.

But i have heared that JBOSS clustering over TCP increases n/w traffic and cannot be scaled beyond 6 servers or so...

Is there a way to over come this issue? like reducing the bandwidth traffic used b/w the clusters or any other other ideas?

Regards

Antonio

[brian.stansberry]

TCP definitely doesn't scale as well as UDP multicast. But whether that is a real issue depends on what kind of traffic you have between the nodes. The services that generate a lot of traffic over the JGroups channels are

1) HttpSession replication
2) EJB StatefulSessionBean replication
3) Hibernate/JPA entity caching.

1) and 2) can be made highly scalable by using buddy replication. 3) is more complex, but there are things you can do as well (use INVALIDATION_SYNC, don't cache query result sets, don't cache entities that are frequently updated.)