Help me understand Balancers and multiple EAP clusters

[ehle]

Hello,

I've read through the docs, but I'm still not sure I grok how balancers work in relation to EAP6/AS7(WF7?) clusters, and EAP mod_cluster groups - especially when you have multiple ones.

We have been very successful with "simple" environment  of 2 web front end servers, behind a hardware load balancer,  supporting several JBoss instances.

Apache mod_cluster config looks like this (on both systems):

mod_cluster.conf:

LoadModule proxy_cluster_module modules/mod_proxy_cluster.so

LoadModule slotmem_module modules/mod_slotmem.so

LoadModule manager_module modules/mod_manager.so

LoadModule advertise_module modules/mod_advertise.so

MemManagerFile /var/cache/mod_cluster

# Start ModCluster/Jboss Config

<IfModule manager_module>

   Listen 6666

   <VirtualHost *:6666>

     ServerName jboss-www-alpha.my.domain:6666

     ServerAlias *:6666

     <Directory />

       Order deny,allow

       #Deny from all

       #Allow from all

       Allow from 192.168.1.0/24

       Allow from 127.0.0.1

     </Directory>

    EnableMCPMReceive On

    ManagerBalancerName alpha_cluster

    AdvertiseGroup 224.0.2.10:23364

    AdvertiseFrequency 5

    ServerAdvertise On

    KeepAliveTimeout 60

    MaxKeepAliveRequests 0

    <Location /mod_cluster_manager>

      SetHandler mod_cluster-manager

      Order deny,allow

      AllowDisplay on

      Deny from all

      # Allow from all

      Allow from 192.168.1.0/24

   </Location>

  </VirtualHost>

</IfModule>

# End ModCluster/Jboss Config

And a  virtual host entry that contains:

    ProxyPass        / balancer://alpha_cluster stickysession=JSESSIONID|jsessionid nofailover=On

    ProxyPassReverse / balancer://alpha_cluster

JBoss Config:

        <subsystem xmlns="urn:jboss:domain:modcluster:1.1">

            <mod-cluster-config advertise-socket="modcluster" proxy-list="192.198.1.20:6666,192.198.1.25:6666" proxy-url="/" balancer="alpha_cluster" advertise="true" connector="ajp">

                <dynamic-load-provider>

                    <load-metric type="busyness"/>

                </dynamic-load-provider>

            </mod-cluster-config>

        </subsystem>

That all works

The jboss stuff all identifies itself as alpha_cluster in its eap and mod_cluster cluster config info.

However I was trying to figure out, what we would need to do if we add another eap cluster? Especially if we wanted to do rolling deployment, or otherwise be able to stop new connections going to all members of the cluster at once. 

I thought we would need to create a different mod_cluster grouping for the second beta_cluster:

        <subsystem xmlns="urn:jboss:domain:modcluster:1.1">

            <mod-cluster-config advertise-socket="modcluster" proxy-list="192.198.1.20:6666,192.198.1.25:6666" proxy-url="/" balancer="beta_cluster" advertise="true" connector="ajp">

                <dynamic-load-provider>

                    <load-metric type="busyness"/>

                </dynamic-load-provider>

            </mod-cluster-config>

        </subsystem>

and if we do that, the mod_cluster_manager page does see the second Balancer  and lists it.

But at that point I'm kind of lost...

Will we need another set of ProxyPass entries (as below) or does mod_cluster create the extra balancers based on the fact that there are EAP clusters telling it that they are needed?

    ProxyPass        / balancer://beta_cluster stickysession=JSESSIONID|jsessionid nofailover=On

    ProxyPassReverse / balancer://beta_cluster

Will we need to make changes to your mod_cluster.conf?

I could be wrong, but it it seemed that the 2 mod_cluster systems were not forming a "domain" and I did not see the option to disable all the alpha_cluster system contexts in the mod_cluster_manager page - is there something else we need for that?

If we have a second balancer listed in the same virtual host entry... will there be context conflicts? 

What is alpha_cluster has myapp version 0.2 at /foo/myapp and beta_cluster has myapp version 1.3 at /foo/myapp ?

What happens if alpha_cluster has alpha_app at /foo/alpha_app and beta_cluster has beta_app at /foo/beta_app?

beta_cluster ONLY has /foo/beta_app and alpha ONLY hass /foo/alpha_app  as requests come in for http://mysite.com/foo/alpha_app and http://mysite.com/foo/beta_app, the apache server/mod_cluster will do the sorting of which server to send the request to automagically correct?

Sorry if these are really basic questions... I just wasn't having much luck with seeing full multi-balancer examples (though this is pretty close http://httpd.apache.org/docs/2.2/mod/mod_proxy.html ) and wanted to see if I could get a better understanding.

Thanks in advance!

David.

[jfclere]

You can have a bunch of balancers the problem is how to route  /foo/myapp to one or the other AS.

you need to have virtual hosts in httpd or in AS (with different aliases).

[ehle]

Thank you Jean-Frederic, sorry for the slow response.

Maybe I should break my questions down further

1.  IF using  Unique Contexts/AppNames - /foo/foo_app will ONLY be on jboss_clusterA , and AppName/Context /foo/bar_app will only be on jboss_clusterB-

I will only need one Balancer that can serve both clusters because  mod_cluster will keep track of which systems say they have /foo/foo_app, which say they have /foo/bar_app and only send traffic to jboss instances that say the can support it.  Same for /foo/foo_app + /bar/foo_app, and any other case where VirtualHost+context+app is unique

Is that correct? If not what am I getting wrong?

2. IF >1 EAP clusters have collisions of Context/AppName - /foo/foo_app is on BOTH jboss_clusterA AND jboss_clusterB -

Then then unique virtual hosts will be required to distinguish between them (Ex: http://siteA.com/foot/foo_app and http://siteB.com/foot/foo_app).

Even if I have multiple balancers,

Is that correct? If not what am I getting wrong?

3. In a case of "rolling deployments" where Context/AppName collision is intentional,  I could force NEW raffic from the jboss_clusterA to jboss_clusterB (both with same contexts & app names),

And then tell EACH mod_cluster instance to disable each context on jboss_clusterA. ModCluster will stop sending new traffic to jboss_clusterA, and when all existing connections are done, jboss_clusterA can go down for maintenance, new deploment versions, whatever.

Is that correct? If not what am I getting wrong?

4. So I'm not sure what I would be getting by setting up multiple balancers ex:

    ProxyPass        / balancer://A_cluster stickysession=JSESSIONID|jsessionid nofailover=On

    ProxyPassReverse / balancer://A_cluster

    ProxyPass        / balancer://B_cluster stickysession=JSESSIONID|jsessionid nofailover=On

    ProxyPassReverse / balancer://B_cluster

Is it just be that I could configure the balancers differently (what metric to balance on for example)?

5. Some documentation I've read seems to indicate that instead of disabling each context individually, or each node individually, if all nodes in jboss_clusterA are using Balancer://A_cluster and all nodes in jboss_clusterB are using Balancer://B_cluster you could go to  https://server_hosting_siteA.com:6666/mod_cluster_manager and disable all contexts managed by Balancer://A_cluste or  Balancer://B_cluster.

Is that true?  My (incomplete) tests have not shown me that option at the  https://server_hosting_siteA.com:6666/mod_cluster_manage page... is there something else I need to add to the configuration above?

Thanks again for your patience with the huge walls of text

David.

[jfclere]

1) yes the alias and context must be unique. I don't understand what is your VirtualHost+context+app what is app?

2) no the UseAlias directive will route separatly http://siteA.com/foot and http://siteB.com/foot (still not sure what you meant by foo_app? Is it a servlet?

3) you can use the /mod_cluster_manager page or the CLI to disable and later stop the context on a node (the new node must have the same Alias) in case the 2 nodes are no clusterized. If they are in a cluster there is nothing to do mod_cluster wil handled the redeploy or undeploy automatically.

4) that syntax is invalid, mod_cluster has only one logic on httpd side the load factor must be defined on the AS side.

5) you can't disable/stop/enable all nodes of a balancer.

[ehle]

Jean,

Responses in line:

Jean-Frederic Clere wrote:

1) yes the alias and context must be unique. I don't understand what is your VirtualHost+context+app what is app?

   OK Alias = "Stuff Left of path to resource"

        Context = " Stuff to the right of Alias"

"app" is just Application_Name - I think I'm using terminolgy wrong, and should just be calling the everything to the right of protocol://host:<port>/ context - sorry new to this part of IT.

2) no the UseAlias directive will route separatly http://siteA.com/foot and http://siteB.com/foot (still not sure what you meant by foo_app? Is it a servlet?

    OK, I think I got that then - you can set up multiple different ways to "get to" a given context via aliases, apache virtual hosts, jboss virtual hosts and combinations,  and to keep the Alias+Context from colliding across JBoss Clusters.

    Single vs multiple mod_cluster Balancers is extraneous to all of that.

3) you can use the /mod_cluster_manager page or the CLI to disable and later stop the context on a node

    node= server of one or more contexts - for my situation a standalone jboss instance

(the new node must have the same Alias)

    i.e it can be reached by requests using the same uri - Ex: if  http://siteA.com/foot was previously served by nodeA, nodeB will also have to be accessible with the same address

in case the 2 nodes are not clusterized. 

     Here is where I'm losing you...  when you say cluster -  you mean JBoss Cluster (HA/Applicationn state Persistance/Caching)? 

If they are in a cluster there is nothing to do mod_cluster will handled the redeploy or undeploy automatically.

     This is where I'm getting really confused...  I thought if mod_cluster "sees" or "is told" that  http://siteA.com/foot is available on nodeA, nodeB, and nodeC it would distribute requests across all 3 based on the load balancing alogrithim its given.  If its no longer available on nodeA for whatever reason (context disabled via /mod_cluster_manager or CLI, context disabled/undployed in jboss on nodeA, server goes down,etc)  I though mod_cluster would just stop sending it traffic and distribute to remaining nodes that can handle it.         Are you just saying that between nodes in a JBoss cluster, this transition should be graceful?

    Mod_cluster doesn't "know" about what JBoss Cluster a node is in, right? I thought it just new what balancer group the nodes said they were in.

4) that syntax is invalid, mod_cluster has only one logic on httpd side the load factor must be defined on the AS side.

    I think this goes back to my original problem - I really don't understand the why and how of using multiple balancers.

    What ARE the use case for multiple Balancers?  Where/How would you configure them?

5) you can't disable/stop/enable all nodes of a balancer.

   Thank you, I think that was an idea I got from 3rd party (not RH/Jboss.org) that was just plain wrong.

So I think you have dispelled a lot of my misconceptions about places where I THOUGHT we would need an additional Balancers. I think the thing I'm really just not understanding now is When/If you WOULD want to use more than one Balancer.

Is it only when you are in case where, for instance, you want to use the same web front end/proxy for mutiple incompatible backends, or backends that require different mod_proxy tuning?

Could you expand my example from above to show how /where you would define a second balancer if you needed one?

Thanks again for your help!

David.

[jfclere]

3) yes the transition should be graceful. If you have several clusters you can group nodes using domain. Failover occurs in domain first then accross domains.

4) multiple balancers are for multiple ProxPass/ProxyPassMatch or multi VirtualHost in httpd. The feature is there to help migration from mod_proxy_balancer to mod_cluster.

5) normally you don't need a multiple balancers.

[ehle]

Thank you Jean-Frederic,

I think that really clarified things!

Much appreciated.