Seperating Virtual Hosts and Clusters
This is a discussion I started a while ago in the Beginners section (since I am a beginner). Seems that the question isn't that simple, and so I am reposting in this community in the hope to get an anwer (see also http://community.jboss.org/message/577472#577472).
In this post I have done a slightly different configuration (still not working, though):
The thing I am trying to achieve with mod_cluster, is describes as follows:
We have three seperate servers in our setup:
192.168.0.1 webserver Apache 2.2.x
192.168.0.2 appserver1 JBoss-AS 5
192.168.0.3 appserver2 JBoss-AS 5
We configured the DNS such that www.application1.nl and www.application2.nl are landing on the webserver. The aim is to get application contexts /application1 and /application2 mapped to there own cluster:
- cluster1 contains JBoss instances application1_i1, application1_i2, running on respectively appserver1 and appserver2, and reacheable via http://www.application1.nl, which proxies though to ajp://appserver1:8009/application1 and ajp://appserver2:8009/application1
- cluster2 contains JBoss instances application2_i1, application2_i2, running on respectively appserver1 and appserver2, and reacheable via http://www.application2.nl. which proxies though to ajp://appserver1:8109/application1 and ajp://appserver2:8109/application1
The first question is. is this possible (notice I am using the same IP-address, the onlky thing that differs is the ajp port that is used [or do you have to choose a different IP-address for each cluster in this case].
Webserver Confguration:
cluster.conf (in our case put in the conf.d/ directory and loaded into Apache via an Include):
=================================================================
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
LoadModule slotmem_module modules/mod_slotmem.so
LoadModule manager_module modules/mod_manager.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
LoadModule advertise_module modules/mod_advertise.so
LoadModule rewrite_module modules/mod_rewrite.so
CreateBalancers 0
ServerAdvertise Off
UseAlias 1
End of httpd.conf:
============
NameVirtualHost *:80
#
# Include various virtual host configurations
#
Include "conf/virtual_hosts/jboss.conf"
Include "conf/virtual_hosts/application1conf"
Include "conf/virtual_hosts/application2.conf"
jboss.conf:
=======
#
# Default virtual host
#
<VirtualHost *:80>
ServerAdmin a@b.c
DocumentRoot /apps/webserver/data
ServerName webserver:80
ErrorLog logs/nlptc27b14-error_log
CustomLog logs/nlptc27b14-access_log combined
LogLevel debug
</VirtualHost>
#
# Backtrack Listeners
#
Listen webserver:7000
Listen webserver:7001
#
# Backtrack for cluster1
#
<VirtualHost webserver:7000>
ServerAdmin a@b.c
DocumentRoot /apps/webserver/data
ServerName webserver:7000
ErrorLog logs/nlptc27b14-7000-error_log
CustomLog logs/nlptc27b14-7000-access_log combined
LogLevel debug
</VirtualHost>
#
# Backtrack for cluster1
#
<VirtualHost webserver:7001>
ServerAdmin a@b.c
DocumentRoot /apps/webserver/data
ServerName webserver:7001
ErrorLog logs/nlptc27b14-7001-error_log
CustomLog logs/nlptc27b14-7001-access_log combined
LogLevel debug
</VirtualHost>
So in this case, each cluster has it's own port to communicate back (it is this port we edit in the jboss.proxyList in the application server instance). Using UseAlias 1 means that we have to add an <Alias>www.application1.nl</Alias> tag to the Host entry in the server.xml of the application server instance.
application1.conf:
<VirtualHost *:80>
ServerAdmin a@b.c
DocumentRoot /apps/www.application1.nl/data
ServerName www.application1.nl:80
ErrorLog logs/www.application1.nl-error_log
CustomLog logs/www.application1.nl-access_log combined
LogLevel debug
ProxyPass / balancer://cluster1/application1 stickysession=JSESSIONID|jsessionid nofailover=On
ProxyPassReverse / balancer://cluster1/application1
</VirtualHost>
application2.conf:
<VirtualHost *:80>
ServerAdmin a@b.c
DocumentRoot /apps/www.application2.nl/data
ServerName www.application2.nl:80
ErrorLog logs/www.application2.nl-error_log
CustomLog logs/www.application2.nl-access_log combined
LogLevel debug
ProxyPass / balancer://cluster2/application2 stickysession=JSESSIONID|jsessionid nofailover=On
ProxyPassReverse / balancer://cluster2/
</VirtualHost>
Appserver1 Configuration of instance application1_i1:
We have created application1_i1 by cloning the production instance which was installed as a default when we installed JBoss 5 (I have understood that the production instance comes with the RedHat version, and is a performance tuned instance which is analogous to the all instance in Community JBoss 5 servers). In this new instance, we have deployed the following:
- application1.war
- mod-cluster.sar
Configuration of mod-cluster.sar :
In the mod-cluster.sar/META-INF/mod-cluster-jboss-beans.xml we added / changed the following in the HAModClusterConfig section:
<!-- Configuration values for the load balancer itself (must be the
same on all nodes in the cluster). These will be passed to the
load balancer. -->
<property name="stickySession">true</property>
<property name="stickySessionForce">true</property>
<property name="stickySessionRemove">false</property>
<property name="maxAttempts">2</property>
<property name="workerTimeout">-1</property>
<property name="balancer">cluster1</property>
During startup, we enter the proxyList as -Djboss.modcluster.proxyList="webserver:7000", where 7000 matches the port we had in the jboss.conf (the other cluster has another (backtrack) port).
Configuration of jboosweb.sar:
In the jbossweb.sar/server.xml we added the following things:
a. Added the following listener:
<Listener className="org.jboss.web.tomcat.service.deployers.MicrocontainerIntegrationLifecycleListener" delegateBeanName="HAModClusterService"/>
b. added jvmRoute="${jboss.jvmRoute}" to the jboss.web Engine entry
The variable ${jboss.jvmRoute} is given during startup via a -D option.
c. added <Alias>www.application1.nl</Alias> after the <Host name="localhost"> entry to accomodate UseAlias 1
(the rest of the appservers and instances are configured and installed in an analogous way. On appserver1, we have instance application2_i1, which is the same as application1_i1. The only difference with application1_i2 is that we deploy application2 in this instance. Of course the Alias entry is also modified).
Startup (example):
On appserver1, we can now start (for example) instance application1_i1 as follows:
cd /opt/jboss/jboss-as/bin; /opt/jboss/jboss-as/bin/run.sh -c application1_i1 -b appserver1 -g cluster1 -u 239.255.100.100 -m 60100 -Djboss.messaging.ServerPeerID=0 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/logs/ -Djboss.jvmRoute="appserver1" -Djboss.Domain=cluster1 -Djboss.modcluster.proxyList="192.168.0.1:7000"
On appserver2, we can now start (for example) instance application1_i2 as follows:
cd /opt/jboss/jboss-as/bin; /opt/jboss/jboss-as/bin/run.sh -c application1_i2 -b appserver2 -g cluster1 -u 239.255.100.100 -m 60100 -Djboss.messaging.ServerPeerID=1 -Djboss.service.binding.set=ports-default -Djboss.server.log.dir=/apps/logs/ -Djboss.jvmRoute="appserver1" -Djboss.Domain=cluster1 -Djboss.modcluster.proxyList="192.168.0.1:7000"
Difficulty:
I can start one application cluster and include one virtual host, and in fact http://www.application1.nl/application1 works with all bells and whistles. The problem starts when I start the other cluster and include the other virtual host. The problem is I think that the www.application1.nl still communicates with both clusters:
cat www.application1.nl-error_log | grep ajp | grep 8 | sort -u
...
[Wed Dec 29 09:10:02 2010] [debug] mod_proxy_cluster.c(232): Created: reusing worker for ajp://192.168.0.2:8009
[Wed Dec 29 09:10:02 2010] [debug] mod_proxy_cluster.c(381): Created: worker for ajp://192.168.0.2:8009 1 (status): 1
[Wed Dec 29 09:10:13 2010] [debug] mod_proxy_cluster.c(232): Created: reusing worker for ajp://192.168.0.2:8109
[Wed Dec 29 09:10:13 2010] [debug] mod_proxy_cluster.c(381): Created: worker for ajp://192.168.0.2:8109 2 (status): 1
...
which is fine when I had deployed both of the application to each instance. The reason we don't want that is because the owners's and QOS of application1 can be totally different than of application2, and we want to monitor and maintain them seperately.
Questions:
- Is the architecture in the beginning of the post achievable with mod_cluster.so (it is in mod_jk.so but it is resulting in a very long workers.properties, plus you don't have all the "magic" mod_cluster promisses) ? If yes, can someone give a simple configuration example (the minimal configuration in the mod_cluster documentation adds just one virtual host and one cluster, but I want more).
- Can you use the ports-default, ports-01, etc mechanism of JBoss, or do you have to choose a seperate binding IP address for each cluster instance in this case (this means a lot of IP-addresses).
- I there a simple way to seperate cluster traffic (such that only ajp://192.168.0.2:8009 is seen by vhost www.application1.nl, and only ajp://192.168.0.2:8109 is seen by vhost www.application2.nl).
Related Posts:
