3 Replies Latest reply on Jul 11, 2016 8:23 AM by wdfink

How to isolate two mod_cluster balancers within network of nodes

amostech Jul 7, 2016 5:48 PM

Good afternoon, I am having trouble with mod_cluster and mod_advertise.

I have a cluster of 16 virtual machines running JBoss AS7 and they are registered to a mod_cluster load balancer.

This seemed to be working pretty fine. Until when we started a new project and had to allocate another cluster of 4 servers for a new application. This new application is running in machines that share the same network IP address range...

XXX.XXX.XXX.1

XXX.XXX.XXX.2

XXX.XXX.XXX.3

XXX.XXX.XXX.4

XXX.XXX.XXX.5 -> For example, this IP is not mine, since we are in a public cloud I have no control over the allocation of IP to my servers. Will this machine receive messages from my cluster also? For example, if somebody installs a mod_cluster in this machine they will be able to see my servers? And even worse, will they be able to direct calls to my machines through any other application contexts that they might have there?

...

XXX.XXX.XXX.32

==== Here ends my public IP range ===

XXX.XXX.XXX.XXX

When we bootstrap both clusters we are seeing servers from Application 1 in mod-cluster-manager of cluster 2 and vice-versa. I was reading about this and figured that this could be related to the multicast address that is used by jboss to register its presence within mod_cluster / mod_avertise. Is that correct? (Setting multiple mod_cluster load balancers in clustering environment - JBoss AS 7.1 - Project Documentation Editor)

This started to scare me, because those servers are in a public cloud. Meaning they have valid public IP addresses. Does that mean that JBoss is trying to broadcast messages to all other IPs that are within the IP range? This means that I am sending requests to register servers to other machines even to the ones that are in the same IP range but do not belong to my cluster?

I dont know if I am explaining my problem correctly, but I just dont understand how to set this up properly.

Basically here is what I want to get:

But since I am using the same multicast address because this is the one that comes setup by default in JBoss domain.xml this is what I am getting:

Here is how domain.xml comes by default:

<socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>

And whenever I try to change this multicast-address to let`s say: 224.0.1.106. The mod-cluster-manager keeps seeing all the servers that I have mention in the picture.

So in conclusion, I dont really know where to change the multicast-address and if that is the correct way to achieve what I need.

Regards,

1. Re: How to isolate two mod_cluster balancers within network of nodes

wdfink Jul 8, 2016 2:39 AM (in response to amostech)

If I understand your configuration correct, you have one domain and all servers vor mode_cluster1 and mod_cluster2 are using the same profile here, right?
But drive a different application and it should form two separate clusters?
In that case you need to set different multicast addresses for JGroups communication - otherwise all nodes see each other internal
and a different mod_cluster advertizing multicast address.

If you don't want to use separate profiles as all other config is the same you can use expressions and properties (like it is for JGroups multicast)
   <socket-binding name="jgroups-udp" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/>
change the line:
   <socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>
to
   <socket-binding name="modcluster" port="0" multicast-address="${jboss.modcluster.multicast.address:224.0.1.105}" multicast-port="23364"/>

Now you need to override the expression by adding this to the server-group
    <system-properties>
        <property name="jboss.default.multicast.address" value="230.0.0.5" boot-time="true"/>
        <property name="jboss.modcluster.multicast.address" value="224.0.1.106" boot-time="true"/>
    </system-properties>

That should override the multicast addresses for the complete server-group and separate it. Note you need to configure the LB accordingly.
Actions
2. Re: How to isolate two mod_cluster balancers within network of nodes

amostech Jul 8, 2016 10:54 AM (in response to wdfink)

Mr Wolf. Your answer helped a lot. You went right on the spot. The solution to the problem was to specify a different multicast address both on domain.xml at

<socket-binding name="modcluster" port="0" multicast-address="224.0.1.105" multicast-port="23364"/>

and I figured that I was also required to set the AdvertiseGroup Address:Port (multicast set in domain.xml) within the mod_cluster.conf file in apache also.

And just as an FYI in case someone else finds this thread in the future I would like to recommend everyone to ALWAYS use an advertise-security-key while using multicast because when you are in a public cloud you NEVER want other IP addresses that are in the same range as yours to be able to interpret the broadcast messages sent by your slave servers.

To achieve this simply find the modcluster subsystem inside the profile you are using which should be ha or ha-full or any derivation of those.

<subsystem xmlns="urn:jboss:domain:modcluster:1.1">
                <mod-cluster-config advertise-socket="modcluster" advertise-security-key="{PUT A SECURITY KEY HERE}" load-balancing-group="${xxxxxxx.modcluster.lbgroup:StdLBGroup}" connector="ajp">
                    <dynamic-load-provider>
                        <load-metric type="busyness"/>
                    </dynamic-load-provider>
                </mod-cluster-config>
            </subsystem>

and within the mod_cluster.conf file in Apache you should end adding the following Tags

     AdvertiseGroup multicast_ip:multicast_port
     AdvertiseSecurityKey {SECURITY KEY THAT YOU DEFINED IN DOMAIN.XML}

Hope this helps someone else.

Best regards,
1 of 1 people found this helpful
Actions
3. Re: How to isolate two mod_cluster balancers within network of nodes

wdfink Jul 11, 2016 8:23 AM (in response to amostech)

Thanks Artur for the final summary and hints, this will definitively help others
Actions

Go to original post