11 Replies Latest reply on Feb 7, 2011 12:10 PM by Rafael Soares (Tuelho)

    Seperating Virtual Hosts and Clusters

    Willem Noorduin Newbie

      This is a discussion I started a while ago in the Beginners section (since I am a beginner). Seems that the question isn't that simple, and so I am reposting in this community in the hope to get an anwer (see also http://community.jboss.org/message/577472#577472).

       

      In this post I have done a slightly different configuration (still not working, though):

       

      The thing I am trying to achieve with mod_cluster, is describes as follows:

       

      We have three seperate servers in our setup:

      192.168.0.1 webserver     Apache 2.2.x
      192.168.0.2 appserver1     JBoss-AS 5
      192.168.0.3 appserver2     JBoss-AS 5

       

      We configured the DNS such that www.application1.nl and www.application2.nl are landing on the webserver. The aim is to get application contexts /application1 and /application2 mapped to there own cluster:

       

      • cluster1 contains JBoss instances application1_i1, application1_i2, running on respectively appserver1 and appserver2, and reacheable via http://www.application1.nl, which proxies though to ajp://appserver1:8009/application1 and ajp://appserver2:8009/application1
      • cluster2 contains JBoss instances application2_i1, application2_i2, running on respectively appserver1 and appserver2, and reacheable via http://www.application2.nl. which proxies though to ajp://appserver1:8109/application1 and ajp://appserver2:8109/application1

       

      The first question is. is this possible (notice I am using the same IP-address, the onlky thing that differs is the ajp port that is used [or do you have to choose a different IP-address for each cluster in this case].

       

      Webserver Confguration:

       

      cluster.conf (in our case put in the conf.d/ directory and loaded into Apache via an Include):

      =================================================================

       

      LoadModule proxy_module modules/mod_proxy.so
      LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
      LoadModule slotmem_module modules/mod_slotmem.so
      LoadModule manager_module modules/mod_manager.so
      LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
      LoadModule advertise_module modules/mod_advertise.so
      LoadModule rewrite_module modules/mod_rewrite.so

       

      CreateBalancers 0
      ServerAdvertise Off
      UseAlias 1

       

      End of httpd.conf:

      ============

       

      NameVirtualHost *:80

       

      #
      # Include various virtual host configurations
      #

       

      Include "conf/virtual_hosts/jboss.conf"
      Include "conf/virtual_hosts/application1conf"
      Include "conf/virtual_hosts/application2.conf"

       

      jboss.conf:

      =======

       

      #
      # Default virtual host
      #

       

      <VirtualHost *:80>
          ServerAdmin a@b.c
          DocumentRoot /apps/webserver/data
          ServerName webserver:80
          ErrorLog logs/nlptc27b14-error_log
          CustomLog logs/nlptc27b14-access_log combined
          LogLevel debug
      </VirtualHost>

       

      #
      # Backtrack Listeners
      #

       

      Listen webserver:7000
      Listen webserver:7001

       

      #
      # Backtrack for cluster1
      #

       

      <VirtualHost webserver:7000>
          ServerAdmin a@b.c

          DocumentRoot /apps/webserver/data
          ServerName webserver:7000
          ErrorLog logs/nlptc27b14-7000-error_log
          CustomLog logs/nlptc27b14-7000-access_log combined
          LogLevel debug
      </VirtualHost>

       

      #
      # Backtrack for cluster1
      #

       

      <VirtualHost webserver:7001>
          ServerAdmin a@b.c

          DocumentRoot /apps/webserver/data
          ServerName webserver:7001
          ErrorLog logs/nlptc27b14-7001-error_log
          CustomLog logs/nlptc27b14-7001-access_log combined
          LogLevel debug
      </VirtualHost>

       

      So in this case, each cluster has it's own port to communicate back (it is this port we edit in the jboss.proxyList in the application server instance). Using UseAlias 1 means that we have to add an <Alias>www.application1.nl</Alias> tag to the Host entry in the server.xml of the application server instance.

       

      application1.conf:

       

      <VirtualHost *:80>
          ServerAdmin a@b.c

          DocumentRoot /apps/www.application1.nl/data
          ServerName www.application1.nl:80
          ErrorLog logs/www.application1.nl-error_log
          CustomLog logs/www.application1.nl-access_log combined
          LogLevel debug

          ProxyPass / balancer://cluster1/application1 stickysession=JSESSIONID|jsessionid nofailover=On
          ProxyPassReverse / balancer://cluster1/application1

      </VirtualHost>

       

      application2.conf:

       

      <VirtualHost *:80>
          ServerAdmin a@b.c

          DocumentRoot /apps/www.application2.nl/data
          ServerName www.application2.nl:80
          ErrorLog logs/www.application2.nl-error_log
          CustomLog logs/www.application2.nl-access_log combined
          LogLevel debug

          ProxyPass / balancer://cluster2/application2 stickysession=JSESSIONID|jsessionid nofailover=On
          ProxyPassReverse / balancer://cluster2/

      </VirtualHost>

       

      Appserver1 Configuration of instance application1_i1:

       

      We have created application1_i1 by cloning the production instance which was installed as a default when we installed JBoss 5 (I have understood that the production instance comes with the RedHat version, and is a performance tuned instance which is analogous to the all instance in Community JBoss 5 servers). In this new instance, we have deployed the following:

       

      • application1.war
      • mod-cluster.sar

       

      Configuration of mod-cluster.sar :

       

      In the mod-cluster.sar/META-INF/mod-cluster-jboss-beans.xml we added / changed the following in the HAModClusterConfig section:

       

      <!-- Configuration values for the load balancer itself (must be the
               same on all nodes in the cluster). These will be passed to the
               load balancer. -->
          <property name="stickySession">true</property>
          <property name="stickySessionForce">true</property>
          <property name="stickySessionRemove">false</property>
          <property name="maxAttempts">2</property>
          <property name="workerTimeout">-1</property>
          <property name="balancer">cluster1</property>

      During startup, we enter the proxyList as -Djboss.modcluster.proxyList="webserver:7000", where 7000 matches the port we had in the jboss.conf (the other cluster has another (backtrack) port).

       

      Configuration of jboosweb.sar:

       

      In the jbossweb.sar/server.xml we added the following things:

       

      a. Added the following listener:

         <Listener className="org.jboss.web.tomcat.service.deployers.MicrocontainerIntegrationLifecycleListener" delegateBeanName="HAModClusterService"/>

       

      b. added jvmRoute="${jboss.jvmRoute}" to the jboss.web Engine entry

      The variable ${jboss.jvmRoute} is given during startup via a -D option.

       

      c. added <Alias>www.application1.nl</Alias> after the <Host name="localhost"> entry to accomodate UseAlias 1

       

      (the rest of the appservers and instances are configured and installed in an analogous way. On appserver1, we have instance application2_i1, which is the same as application1_i1. The only difference with application1_i2 is that we deploy application2 in this instance. Of course the Alias entry is also modified).

      Startup (example):

       

      On appserver1, we can now start (for example) instance application1_i1 as follows:

       

      cd /opt/jboss/jboss-as/bin; /opt/jboss/jboss-as/bin/run.sh -c application1_i1 -b appserver1 -g cluster1 -u 239.255.100.100 -m 60100 -Djboss.messaging.ServerPeerID=0 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/logs/ -Djboss.jvmRoute="appserver1" -Djboss.Domain=cluster1 -Djboss.modcluster.proxyList="192.168.0.1:7000"

       

      On appserver2, we can now start (for example) instance application1_i2 as follows:

       

      cd /opt/jboss/jboss-as/bin; /opt/jboss/jboss-as/bin/run.sh -c application1_i2 -b appserver2 -g cluster1 -u 239.255.100.100 -m 60100 -Djboss.messaging.ServerPeerID=1 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/logs/ -Djboss.jvmRoute="appserver1" -Djboss.Domain=cluster1 -Djboss.modcluster.proxyList="192.168.0.1:7000"

       

      Difficulty:

       

      I can start one application cluster and include one virtual host, and in fact http://www.application1.nl/application1 works with all bells and whistles. The problem starts when I start the other cluster and include the other virtual host. The problem is I think that the www.application1.nl still communicates with both clusters:

       

      cat www.application1.nl-error_log | grep ajp | grep 8 | sort -u

       

      ...

      [Wed Dec 29 09:10:02 2010] [debug] mod_proxy_cluster.c(232): Created: reusing worker for ajp://192.168.0.2:8009
      [Wed Dec 29 09:10:02 2010] [debug] mod_proxy_cluster.c(381): Created: worker for ajp://192.168.0.2:8009 1 (status): 1
      [Wed Dec 29 09:10:13 2010] [debug] mod_proxy_cluster.c(232): Created: reusing worker for ajp://192.168.0.2:8109
      [Wed Dec 29 09:10:13 2010] [debug] mod_proxy_cluster.c(381): Created: worker for ajp://192.168.0.2:8109 2 (status): 1
      ...

       

      which is fine when I had deployed both of the application to each instance. The reason we don't want that is because the owners's and QOS of application1 can be totally different than of application2, and we want to monitor and maintain them seperately.

       

      Questions:

       

      1. Is the architecture in the beginning of the post achievable with mod_cluster.so (it is in mod_jk.so but it is resulting in a very long workers.properties, plus you don't have all the "magic" mod_cluster promisses) ? If yes, can someone give a simple configuration example (the minimal configuration in the mod_cluster documentation adds just one virtual host and one cluster, but I want more).
      2. Can you use the ports-default, ports-01, etc mechanism of JBoss, or do you have to choose a seperate binding IP address for each cluster instance in this case (this means a lot of IP-addresses).
      3. I there a simple way to seperate cluster traffic (such that only  ajp://192.168.0.2:8009 is seen by vhost www.application1.nl, and only ajp://192.168.0.2:8109 is seen by vhost www.application2.nl).

       

      Related Posts:

       

        • 1. Re: Seperating Virtual Hosts and Clusters
          Jean-Frederic Clere Master

          a - you are using "ProxyPass / balancer:... in VirtualHost" and "CreateBalancers 0 / UseAlias 1".

          Both are doing the same thing you should use only one for example "ProxyPass / balancer:... in VirtualHost" and "CreateBalancers 1 / UseAlias 0" or remove the "ProxyPass / balancer:... in VirtualHost" and trust mod_cluster to do the right routing.

           

          b - you are using the same -Djboss.jvmRoute="appserver1" for 2 different nodes that can't work.

           

          1 - yes

          2 - you can use different ports.

          3 - you see traffic on both because of a.

          • 2. Seperating Virtual Hosts and Clusters
            Willem Noorduin Newbie

            Thanx for your help.

             

            When I use the format

             

            • CreateBalancers 1 / UseAlias 0
            • ProxyPass / balancer://cluster1/application1
            • ProxyPassReverse / balancer://cluster1/application1

             

            This leads to a "Service Temporary unavailable" error when I go to the servername in my browser. When I loook into the (debug) errorlog, I see "proxy: byrequest balancer FAILED" and balancer://cluster1: All workers are in error state. So:

             

            # cat www.application1.nl-error_log
            [Mon Jan 10 13:13:41 2011] [debug] mod_proxy_cluster.c(604): update_workers_node starting
            [Mon Jan 10 13:13:41 2011] [debug] mod_proxy_cluster.c(623): update_workers_node done
            ...
            [Mon Jan 10 13:13:41 2011] [debug] mod_proxy_cluster.c(604): update_workers_node starting
            [Mon Jan 10 13:13:41 2011] [debug] mod_proxy_cluster.c(623): update_workers_node done
            [Mon Jan 10 13:14:22 2011] [debug] mod_proxy_cluster.c(1137): proxy: byrequests balancer FAILED
            [Mon Jan 10 13:14:22 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state


            When I use

             

            • CreateBalancer 0 / UseAlias 1,
            • no ProxyPass and ProxyPassReverse statements
            • <Alias>www.application1.nl</Alias> (for the first application) in the server.xml of the jbossweb.sar/ of the JBoss-AS which is serving that application,

             

            I get a 404 Not Found when I go to the www.application1.nl/application1 application root. Now:

             

            # cat www.application1.nl-error_log
            [Mon Jan 10 13:25:05 2011] [debug] mod_proxy_cluster.c(604): update_workers_node starting
            [Mon Jan 10 13:25:05 2011] [debug] mod_proxy_cluster.c(623): update_workers_node done
            ...
            [Mon Jan 10 13:25:05 2011] [debug] mod_proxy_cluster.c(604): update_workers_node starting
            [Mon Jan 10 13:25:05 2011] [debug] mod_proxy_cluster.c(623): update_workers_node done
            [Mon Jan 10 13:25:14 2011] [error] [client 145.78.159.234] File does not exist: /apps/www.application1.nl/data/application1

             

            When I make a application1 subdir (as sometimes was needed in case of mod_jk and JkMount), I get a 403 Forbidden error. So its seems like the loadbalancer wasn't really created..

             

            In both cases:

             

            # cat error_log
            [Mon Jan 10 13:25:04 2011] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
            [Mon Jan 10 13:25:04 2011] [notice] Digest: generating secret for digest authentication ...
            [Mon Jan 10 13:25:04 2011] [notice] Digest: done
            [Mon Jan 10 13:25:04 2011] [error] mod_advertise: ServerAdvertise Address or Port not defined, Advertise disabled!!!
            [Mon Jan 10 13:25:04 2011] [notice] Apache/2.2.14 (Unix) DAV/2 configured -- resuming normal operations

             

            but since we don't use ServerAdvertise here (you could only use one such directive, and we
            have two clusters), I have ignored it thusfar [maybe mod_cluster want something there].

             

            When I do a ps of my jboss environment, you see:

             

            # ps -ef | grep jboss

            jboss    19205     1  0 12:51 ?        00:00:00 /bin/sh /opt/jboss/jboss-as/bin/run.sh -c node1 -b nlptc27b09 -g cluster1 -u 239.255.100.100 -m 60000 -Djboss.messaging.ServerPeerID=0 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/data/node1/logs/ -Djboss.jvmRoute=node1 -Djboss.Domain=cluster1

             

            jboss    19228     1  0 12:51 ?        00:00:00 /bin/sh /opt/jboss/jboss-as/bin/run.sh -c node2 -b nlptc27b09 -g cluster2 -u 239.255.100.101 -m 60001 -Djboss.messaging.ServerPeerID=1 -Djboss.service.binding.set=ports-01 -Djboss.server.log.dir=/apps/data/node2/logs/ -Djboss.jvmRoute=node2 -Djboss.Domain=cluster2

             

            jboss    19283 19205  2 12:51 ?        00:01:12 /usr/local/jdk1.6.0_21/bin/java -Dprogram.name=run.sh -server -Xms512m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.lang.ClassLoader.allowArraySyntax=true -Djava.net.preferIPv4Stack=true -Djava.endorsed.dirs=/distr/jboss/jboss-as/lib/endorsed -classpath /distr/jboss/jboss-as/bin/run.jar:/distr/jdk1.6.0_21/lib/tools.jar org.jboss.Main -c node1 -b nlptc27b09 -g cluster1 -u 239.255.100.100 -m 60000 -Djboss.messaging.ServerPeerID=0 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/data/node1/logs/ -Djboss.jvmRoute=node1 -Djboss.Domain=cluster1

             

            jboss    19306 19228  2 12:51 ?        00:01:11 /usr/local/jdk1.6.0_21/bin/java -Dprogram.name=run.sh -server -Xms512m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.lang.ClassLoader.allowArraySyntax=true -Djava.net.preferIPv4Stack=true -Djava.endorsed.dirs=/distr/jboss/jboss-as/lib/endorsed -classpath /distr/jboss/jboss-as/bin/run.jar:/distr/jdk1.6.0_21/lib/tools.jar org.jboss.Main -c node2 -b nlptc27b09 -g cluster2 -u 239.255.100.101 -m 60001 -Djboss.messaging.ServerPeerID=1 -Djboss.service.binding.set=ports-01 -Djboss.server.log.dir=/apps/data/node2/logs/ -Djboss.jvmRoute=node2 -Djboss.Domain=cluster2

             

            Please ignore the company specific paths here. In this case, I have two single noded clusters (for a demo), each running a different application (In the real environment, each cluster has of course different nodes scattered around our network).

            • 3. Re: Seperating Virtual Hosts and Clusters
              Jean-Frederic Clere Master

              Do you receive the MCMP elements in httpd (you should have them in error_log (debug)? - you should have CONFIG/ENABLE-APP/STATUS etc.

              If not the httpd doesn't receive the nodes information and mod_cluster can't work if it receives the messages looking in the message should give a clue of what is going wrong.

              • 4. Re: Seperating Virtual Hosts and Clusters
                Willem Noorduin Newbie

                I am indeed not receiving the MCMP elements. What I have done now is define a proxyList back to the webserver (it wasn't defined before). Something is not on yet (some sort of Listener, allthough the mod_cluster documentation for 5.x says that no extra config is needed. Can you give a definition of a listener if it's needed. Now, jboss (one of the clusters) is started as:

                 

                jboss     7940     1  0 11:15 ?        00:00:00 /bin/sh /distr/jboss/jboss-as/bin/run.sh -c node1 -b nlptc27b09 -g cluster1 -u 239.255.100.100 -m 60000 -Djboss.messaging.ServerPeerID=0 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/data/node1/logs/ -Djboss.jvmRoute=node1 - jboss.Domain=cluster1 -Djboss.modcluster.proxyList=nlptc27b09.nlptc27.post.tnt:7000

                 

                jboss     8022  7940 11 11:15 ?        00:01:13 /distr/jdk1.6.0_21/bin/java -Dprogram.name=run.sh -server -Xms512m -Xmx512m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.lang.ClassLoader.allowArraySyntax=true -Djava.net.preferIPv4Stack=true -Djava.endorsed.dirs=/distr/jboss/jboss-as/lib/endorsed -classpath /distr/jboss/jboss-as/bin/run.jar:/distr/jdk1.6.0_21/lib/tools.jar org.jboss.Main -c node1 -b nlptc27b09 -g cluster1 -u 239.255.100.100 -m 60000 -Djboss.messaging.ServerPeerID=0 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/data/node1/logs/ -Djboss.jvmRoute=node1 -Djboss.Domain=cluster1 -Djboss.modcluster.proxyList=nlptc27b09.nlptc27.post.tnt:7000

                 

                for this short setup, we have both the AS and the WS on the same server.

                 

                When I start the AS without the WS (such that port 7000 is closed), I don't get an error (this is telling me that the AS is accepting my proxyList, but not doing a thing with it).

                • 5. Seperating Virtual Hosts and Clusters
                  Paul Ferraro Master

                  What version of mod_cluster are you using?

                  • 6. Seperating Virtual Hosts and Clusters
                    Willem Noorduin Newbie

                    I have some mixed results (though I think I am nearly there). My appologies for the long post:

                     

                    I have added a  <depends>ModClusterService</depends> tag to the jboss-beans.xml (WebServer section) of both nodes (I believe this narrows my mod_cluster version down to 1.0.0 or 1.0.<low>, becasuse I have read that this depends is not needed anymore in later versions. Upgrading mod_cluster is in general a good thing, but I am a little bit cornered by RedHat Support, which supports only the version that I have installed now. Note that this has also a bug in the fact that you must specifify ServerName servername:80 in the Apache virtual host to get apache starting. I see now the following:

                     

                    (Note for the sake of completeness that I have to rearrange the architecture dua to a lack of servers. Both the Apache webserver and the JBoss-AS are running now on the IP-address 192.168.0.1).

                    nlptc27b09-7000-access_log and nlptc27b09-7001-access_log

                    (access loggings of the webserver backtrack, the port (7000 or 7001) is the port used in the proxyList of the JBoss-AS node)

                    ======================================================================

                     

                    192.168.0.1 - - [12/Jan/2011:08:32:18 +0100] "INFO / HTTP/1.0" 200 - "-" "ClusterListener/1.0"
                    192.168.0.1 - - [12/Jan/2011:08:32:18 +0100] "CONFIG / HTTP/1.0" 200 - "-" "ClusterListener/1.0"
                    192.168.0.1 - - [12/Jan/2011:08:32:28 +0100] "STATUS / HTTP/1.0" 200 53 "-" "ClusterListener/1.0"
                    192.168.0.1 - - [12/Jan/2011:08:32:38 +0100] "ENABLE-APP / HTTP/1.0" 200 - "-" "ClusterListener/1.0"
                    192.168.0.1 - - [12/Jan/2011:08:32:48 +0100] "STATUS / HTTP/1.0" 200 53 "-" "ClusterListener/1.0"
                    192.168.0.1 - - [12/Jan/2011:08:32:58 +0100] "STATUS / HTTP/1.0" 200 53 "-" "ClusterListener/1.0"
                    192.168.0.1 - - [12/Jan/2011:08:33:08 +0100] "STATUS / HTTP/1.0" 200 53 "-" "ClusterListener/1.0"

                     

                    error logging nlptc27b09-7000-error_log (part of it):

                    ===================================

                     

                    [Wed Jan 12 08:32:19 2011] [debug] mod_proxy_cluster.c(604): update_workers_node starting

                    [Wed Jan 12 08:32:19 2011] [debug] mod_proxy_cluster.c(623): update_workers_node done

                    [Wed Jan 12 08:32:19 2011] [debug] mod_proxy_cluster.c(604): update_workers_node starting

                    [Wed Jan 12 08:32:19 2011] [debug] mod_proxy_cluster.c(623): update_workers_node done

                    [Wed Jan 12 08:32:28 2011] [debug] mod_manager.c(1432): manager_trans STATUS (/)

                    [Wed Jan 12 08:32:28 2011] [debug] mod_manager.c(1947): manager_handler STATUS (/) processing: "JVMRoute=node1&Load=100"

                    [Wed Jan 12 08:32:28 2011] [debug] proxy_util.c(1999): proxy: ajp: has acquired connection for (192.168.0.1)

                    [Wed Jan 12 08:32:28 2011] [debug] proxy_util.c(2055): proxy: connecting ajp://192.168.0.1:8009/ to 192.168.0.1:8009

                    [Wed Jan 12 08:32:28 2011] [debug] proxy_util.c(2153): proxy: connected ajp://192.168.0.1:8009/ to 192.168.0.1:8009

                    [Wed Jan 12 08:32:28 2011] [debug] proxy_util.c(2308): proxy: ajp: fam 2 socket created to connect to 192.168.0.1

                    [Wed Jan 12 08:32:38 2011] [error] ajp_cping_cpong: apr_socket_recv failed

                    [Wed Jan 12 08:32:38 2011] [debug] mod_proxy_cluster.c(1213): ajp_cping_cpong: Done

                    [Wed Jan 12 08:32:38 2011] [debug] proxy_util.c(2017): proxy: ajp: has released connection for (192.168.0.1)

                    [Wed Jan 12 08:32:38 2011] [debug] mod_manager.c(1988): manager_handler STATUS  OK

                    [Wed Jan 12 08:32:38 2011] [debug] mod_manager.c(1432): manager_trans ENABLE-APP (/)

                    [Wed Jan 12 08:32:38 2011] [debug] mod_manager.c(1947): manager_handler ENABLE-APP (/) processing: "JVMRoute=node1&Alias=localhost,www.applic

                    ation1.nl&Context=%2fapplication1"

                    [Wed Jan 12 08:32:38 2011] [debug] mod_manager.c(1988): manager_handler ENABLE-APP  OK

                    [Wed Jan 12 08:32:48 2011] [debug] mod_manager.c(1432): manager_trans STATUS (/)

                     

                    In the nlptc27b09-7001-error_log log I see something similar, but now with port 8109, node2 and application2 (so it seems to me that the right port is connected to the right JBoss-AS instance.

                     

                    Now:

                     

                    1.0. Situation A:

                     

                    jboss.conf:
                    ==========

                    CreateBalancers 1
                    UseAlias 0

                     

                    application1.conf (example):

                    ===========================

                    <VirtualHost *:80>
                        ServerAdmin unix@tntpost.nl
                        DocumentRoot /apps/www.application1.nl/data
                        ServerName www.application1.nl:80
                        ErrorLog logs/www.application1.nl-error_log
                        CustomLog logs/www.application1.nl-access_log combined
                        LogLevel debug

                        ProxyPass / balancer://cluster1/application1 stickysession=JSESSIONID|jsessionid nofailover=On
                        ProxyPassReverse / balancer://cluster1/application1

                    </VirtualHost>

                     

                    nlptc72b14-7000_error_log:

                     

                    [Wed Jan 12 09:42:37 2011] [debug] mod_manager.c(1988): manager_handler STATUS  OK
                    [Wed Jan 12 09:42:47 2011] [debug] mod_manager.c(1432): manager_trans STATUS (/)
                    [Wed Jan 12 09:42:47 2011] [debug] mod_manager.c(1947): manager_handler STATUS (/) processing: "JVMRoute=node1&Load=73"
                    [Wed Jan 12 09:42:47 2011] [debug] proxy_util.c(1999): proxy: ajp: has acquired connection for (192.168.0.1)
                    [Wed Jan 12 09:42:47 2011] [debug] proxy_util.c(2055): proxy: connecting ajp://192.168.0.1:8009/ to 192.168.0.1:8009
                    [Wed Jan 12 09:42:47 2011] [debug] proxy_util.c(2153): proxy: connected ajp://192.168.0.1:8009/ to 192.168.0.1:8009
                    [Wed Jan 12 09:42:47 2011] [debug] mod_proxy_cluster.c(1213): ajp_cping_cpong: Done
                    [Wed Jan 12 09:42:47 2011] [debug] proxy_util.c(2017): proxy: ajp: has released connection for (192.168.0.1)
                    [Wed Jan 12 09:42:47 2011] [debug] mod_manager.c(1988): manager_handler STATUS  OK


                    Result:

                     

                    www.application1.nl/application1 is working as it should
                    www.application1.nl is giving a 503 error (probably to be circumvented by a clever mod_rewrite).

                    www.application2.nl/application2 is working as it shoud
                    www.application2.nl is giving a 503 error (probably to be circumvented by a clever mod_rewrite).

                     

                    Possible reason is in the www.application2.nl-error_log:

                     

                    [Wed Jan 12 09:51:09 2011] [error] proxy: CLUSTER: (balancer://cluster2). All workers are in error state
                    [Wed Jan 12 09:51:42 2011] [debug] mod_proxy_cluster.c(943): get_balancer_by_node found context /application2
                    [Wed Jan 12 09:51:42 2011] [debug] mod_proxy_cluster.c(996): get_context_host_balancer: balancer balancer://cluster1 not found
                    [Wed Jan 12 09:51:42 2011] [debug] mod_proxy_cluster.c(962): get_balancer_by_node balancer NOT found
                    [Wed Jan 12 09:51:42 2011] [debug] mod_proxy_cluster.c(943): get_balancer_by_node found context /application2
                    [Wed Jan 12 09:51:42 2011] [debug] mod_proxy_cluster.c(996): get_context_host_balancer: balancer balancer://cluster1 not found
                    [Wed Jan 12 09:51:42 2011] [debug] mod_proxy_cluster.c(962): get_balancer_by_node balancer NOT found
                    [Wed Jan 12 09:51:42 2011] [debug] mod_proxy_cluster.c(1137): proxy: byrequests balancer FAILED
                    [Wed Jan 12 09:51:42 2011] [error] proxy: CLUSTER: (balancer://cluster2). All workers are in error state

                     

                    While the Proxy statements for the www.application2.nl virtual host are:

                     

                    # cat application2.conf

                    <VirtualHost *:80>
                        ServerAdmin unix@tntpost.nl
                        DocumentRoot /apps/www.application2.nl/data
                        ServerName www.application2.nl:80
                        ErrorLog logs/www.application2.nl-error_log
                        CustomLog logs/www.application2.nl-access_log combined
                        LogLevel debug

                        ProxyPass / balancer://cluster2/application2 stickysession=JSESSIONID|jsessionid nofailover=On
                        ProxyPassReverse / balancer://cluster2/

                    </VirtualHost>

                     

                    Only the first balancer://cluster1 is created I think, and mod_cluster assumes (I think) that the cluster of the
                    www.application2.nl host is also cluster1, while it should create balancer://cluster2


                    1.1. Situation B:


                    jboss.conf (trusting mod_cluster):
                    ==========

                    CreateBalancers 0
                    UseAlias 1

                     

                    application1.conf (example):
                    ===========================

                     

                    <VirtualHost *:80>
                        ServerAdmin unix@tntpost.nl
                        DocumentRoot /apps/www.application1.nl/data
                        ServerName www.application1.nl:80
                        ErrorLog logs/www.application1.nl-error_log
                        CustomLog logs/www.application1.nl-access_log combined
                        LogLevel debug

                    #   ProxyPass / balancer://cluster1/application1 stickysession=JSESSIONID|jsessionid nofailover=On
                    #   ProxyPassReverse / balancer://cluster1/application1

                    </VirtualHost>


                    In nlptc27b09-7000-error_log We see:

                     

                    [Wed Jan 12 10:10:07 2011] [debug] mod_manager.c(1947): manager_handler CONFIG (/) processing: "JVMRoute=node1&Port=8009&Balancer=cluster1&Host=192.168.0.1&Type=ajp&Maxattempts=2"

                     

                    In nlptc27b09-7001-error_log We see:

                     

                    [Wed Jan 12 10:10:02 2011] [debug] mod_manager.c(1947): manager_handler CONFIG (/) processing: "JVMRoute=node2&Port=8109&Balancer=cluster1&Host=192.168.0.1&Type=ajp&Maxattempts=2"

                     

                    So again the balancer has the same number as application1. Maybe I am mistaken, but I think we have to see two
                    different balancers here (because there are two clusters to connect to), as the following shows:

                     

                    # ps -ef | grep jboss

                    jboss    32602     1  0 10:09 ?        00:00:00 /bin/sh /distr/jboss/jboss-as/bin/run.sh -c node1 -b nlptc27b09 -g cluster1 -u 239.255.100.100 -m 60000 -Djboss.messaging.ServerPeerID=0 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/data/node1/logs/ -Djboss.jvmRoute=node1 -Djboss.Domain=cluster1 -Djboss.modcluster.proxyList=nlptc27b09.nlptc27.post.tnt:7000
                    jboss    32623     1  0 10:09 ?        00:00:00 /bin/sh /distr/jboss/jboss-as/bin/run.sh -c node2 -b nlptc27b09 -g cluster2 -u 239.255.100.101 -m 60001 -Djboss.messaging.ServerPeerID=1 -Djboss.service.binding.set=ports-01 -Djboss.server.log.dir=/apps/data/node2/logs/ -Djboss.jvmRoute=node2 -Djboss.Domain=cluster2 -Djboss.modcluster.proxyList=nlptc27b09.nlptc27.post.tnt:7001

                     

                    In fact, in this situation www.application2.nl/application2 is working well, and the other application not.

                     

                    # tail www.application1.nl-error_log

                    ...

                    [Wed Jan 12 10:17:01 2011] [debug] mod_proxy_cluster.c(1956): cluster: Using route node1
                    [Wed Jan 12 10:17:01 2011] [debug] proxy_util.c(1937): proxy: BALANCER: retrying the worker for (192.168.0.1)
                    [Wed Jan 12 10:17:01 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state for route (node1)
                    [Wed Jan 12 10:17:01 2011] [debug] mod_proxy_cluster.c(1628): cluster: Found value eA5ieqIV6sMq4UO3ptf6oA__.node1 for stickysession JSESSIONID|jsessionid
                    [Wed Jan 12 10:17:01 2011] [debug] mod_proxy_cluster.c(1956): cluster: Using route node1
                    [Wed Jan 12 10:17:01 2011] [debug] proxy_util.c(1937): proxy: BALANCER: retrying the worker for (192.168.0.1)
                    [Wed Jan 12 10:17:01 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state for route (node1)
                    [Wed Jan 12 10:17:02 2011] [debug] mod_proxy_cluster.c(1628): cluster: Found value eA5ieqIV6sMq4UO3ptf6oA__.node1 for stickysession JSESSIONID|jsessionid
                    [Wed Jan 12 10:17:02 2011] [debug] mod_proxy_cluster.c(1956): cluster: Using route node1
                    [Wed Jan 12 10:17:02 2011] [debug] proxy_util.c(1937): proxy: BALANCER: retrying the worker for (192.168.0.1)
                    [Wed Jan 12 10:17:02 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state for route (node1)
                    [Wed Jan 12 10:17:56 2011] [debug] mod_proxy_cluster.c(1628): cluster: Found value eA5ieqIV6sMq4UO3ptf6oA__.node1 for stickysession JSESSIONID|jsessionid
                    [Wed Jan 12 10:17:56 2011] [debug] mod_proxy_cluster.c(1956): cluster: Using route node1
                    [Wed Jan 12 10:17:56 2011] [debug] proxy_util.c(1937): proxy: BALANCER: retrying the worker for (192.168.0.1)
                    [Wed Jan 12 10:17:56 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state for route (node1)


                    # tail www.application2.nl-error_log

                    ...

                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[0] [X-Powered-By] = [Servlet 2.5; JBoss-5.0/JBossWeb-2.1]
                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[1] [Set-Cookie] = [JSESSIONID=X9inOb7SYoHuWY0bufAj+A__.node2; Path=/application2]
                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[2] [Content-Type] = [text/html;charset=ISO-8859-1]
                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(609): ajp_unmarshal_response: ap_set_content_type done
                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[3] [Content-Length] = [1584]
                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 03
                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(697): ajp_parse_type: got 03
                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 05
                    [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(697): ajp_parse_type: got 05
                    [Wed Jan 12 10:17:17 2011] [debug] mod_proxy_ajp.c(562): proxy: got response from (null) (192.168.0.1)
                    [Wed Jan 12 10:17:17 2011] [debug] proxy_util.c(2017): proxy: AJP: has released connection for (192.168.0.1)

                     

                    It seems like the first one gets a valid working loadbalancer. B.t.w. http://192.168.0.1:8080/application1/ and
                    http://192.168.0.1:8180/application2/ are working, demonstrating that the applications are up.

                    • 7. Seperating Virtual Hosts and Clusters
                      Jean-Frederic Clere Master
                      # tail www.application1.nl-error_log

                      ...

                      [Wed Jan 12 10:17:01 2011] [debug] mod_proxy_cluster.c(1956): cluster: Using route node1
                      [Wed Jan 12 10:17:01 2011] [debug] proxy_util.c(1937): proxy: BALANCER: retrying the worker for (192.168.0.1)
                      [Wed Jan 12 10:17:01 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state for route (node1)
                      [Wed Jan 12 10:17:01 2011] [debug] mod_proxy_cluster.c(1628): cluster: Found value eA5ieqIV6sMq4UO3ptf6oA__.node1 for stickysession JSESSIONID|jsessionid
                      [Wed Jan 12 10:17:01 2011] [debug] mod_proxy_cluster.c(1956): cluster: Using route node1
                      [Wed Jan 12 10:17:01 2011] [debug] proxy_util.c(1937): proxy: BALANCER: retrying the worker for (192.168.0.1)
                      [Wed Jan 12 10:17:01 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state for route (node1)
                      [Wed Jan 12 10:17:02 2011] [debug] mod_proxy_cluster.c(1628): cluster: Found value eA5ieqIV6sMq4UO3ptf6oA__.node1 for stickysession JSESSIONID|jsessionid
                      [Wed Jan 12 10:17:02 2011] [debug] mod_proxy_cluster.c(1956): cluster: Using route node1
                      [Wed Jan 12 10:17:02 2011] [debug] proxy_util.c(1937): proxy: BALANCER: retrying the worker for (192.168.0.1)
                      [Wed Jan 12 10:17:02 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state for route (node1)
                      [Wed Jan 12 10:17:56 2011] [debug] mod_proxy_cluster.c(1628): cluster: Found value eA5ieqIV6sMq4UO3ptf6oA__.node1 for stickysession JSESSIONID|jsessionid
                      [Wed Jan 12 10:17:56 2011] [debug] mod_proxy_cluster.c(1956): cluster: Using route node1
                      [Wed Jan 12 10:17:56 2011] [debug] proxy_util.c(1937): proxy: BALANCER: retrying the worker for (192.168.0.1)
                      [Wed Jan 12 10:17:56 2011] [error] proxy: CLUSTER: (balancer://cluster1). All workers are in error state for route (node1)


                      # tail www.application2.nl-error_log

                      ...

                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[0] [X-Powered-By] = [Servlet 2.5; JBoss-5.0/JBossWeb-2.1]
                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[1] [Set-Cookie] = [JSESSIONID=X9inOb7SYoHuWY0bufAj+A__.node2; Path=/application2]
                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[2] [Content-Type] = [text/html;charset=ISO-8859-1]
                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(609): ajp_unmarshal_response: ap_set_content_type done
                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(599): ajp_unmarshal_response: Header[3] [Content-Length] = [1584]
                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 03
                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(697): ajp_parse_type: got 03
                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(687): ajp_read_header: ajp_ilink_received 05
                      [Wed Jan 12 10:17:17 2011] [debug] ajp_header.c(697): ajp_parse_type: got 05
                      [Wed Jan 12 10:17:17 2011] [debug] mod_proxy_ajp.c(562): proxy: got response from (null) (192.168.0.1)
                      [Wed Jan 12 10:17:17 2011] [debug] proxy_util.c(2017): proxy: AJP: has released connection for (192.168.0.1)

                       

                      It seems like the first one gets a valid working loadbalancer. B.t.w. http://192.168.0.1:8080/application1/ and
                      http://192.168.0.1:8180/application2/ are working, demonstrating that the applications are up.

                      You have something wrong with node1 check the AJP connector and the server.log. Which port are you using? A firewall somewhere?

                      • 8. Re: Seperating Virtual Hosts and Clusters
                        Rafael Soares (Tuelho) Novice

                        Hello!

                         

                        I've the same situation. This topic is being very usefull to solve my issue.

                         

                        I think would be nice have this information compiled (using isolated clusters with mod_cluster) in mod_cluster's oficial docs.

                         

                         

                        Good job!

                        • 9. Seperating Virtual Hosts and Clusters
                          Willem Noorduin Newbie

                          I am not a developer, but I am in the middle of writing a document (for my company, but I will keep it general) that describes exactly the situaion for mod_cluster 1.0 (when you need the Enterprise Edition, you don't come any higher) and mod_cluster 1.1.1. Hopefully, after comments from other people (i.e. maybe you can tweak mod_cluster in this way, but maybe we aren't supposed to and we are using something that is scrubbed in the next version, and that kind of thing) we can promote such a topic to a part of the documentation/

                           

                          It would be great to have a best practices guide. I think it would help JBoss to gain more popularity. because then it is even easier to implement.

                          • 10. Seperating Virtual Hosts and Clusters
                            Rafael Soares (Tuelho) Novice

                            Cool! Please post your work in a wiki page on jboss.org when it has finished! ;-)

                             

                            About the issue described in this topic. Have you solved it? If yes, what version of mod_cluster/JBoss have you used?

                            I've tried many scenarios following the config described here and [1], but no succes yet.

                             

                            When I use just one vhost it works fine, but when I try to use 2 vhosts to separate 2 clusters on same box like your scenario I get the same error:

                             

                            "

                            [Mon Feb 07 09:18:45 2011] [debug] mod_proxy_cluster.c(943): get_balancer_by_node balancer NOT found

                            [Mon Feb 07 09:18:45 2011] [debug] mod_proxy_cluster.c(943): get_balancer_by_node balancer NOT found

                            [Mon Feb 07 09:18:45 2011] [debug] mod_proxy_cluster.c(943): get_balancer_by_node balancer NOT found

                            [Mon Feb 07 09:18:45 2011] [debug] mod_proxy_cluster.c(943): get_balancer_by_node balancer NOT found

                            [Mon Feb 07 09:18:45 2011] [debug] mod_proxy_cluster.c(943): get_balancer_by_node balancer NOT found

                            [Mon Feb 07 09:18:45 2011] [debug] mod_proxy_cluster.c(943): get_balancer_by_node balancer NOT found

                            [Mon Feb 07 09:18:45 2011] [debug] mod_proxy_cluster.c(1118): proxy: byrequests balancer FAILED

                            [Mon Feb 07 09:18:45 2011] [error] proxy: CLUSTER: (balancer://cluster-web-01). All workers are in error state

                            "

                            Thanks!

                             

                            [1] http://community.jboss.org/message/577472#577472

                            • 11. Seperating Virtual Hosts and Clusters
                              Rafael Soares (Tuelho) Novice

                              Hello!

                               

                              I tested using the following versions and works fine now:

                               

                              OS: RHEL 5.6 x86

                              JBoss: EAP 5.0.1

                              EWS: 1.0.1 (Apache/2.2.14)

                              mod_cluster: 1.1.1-Final