Global caching
Hello everybody,
I've searching around for the last two weeks trying to find a solution for the following set-up. What I would like to achieve is a set-up where 4 JBoss instances work together in a clustered environment. Two nodes are actually serving client requests while two other nodes are being hot-standby being Apache working as software loadbalancer. This works just great and is up and running. So no difficulties concerning this matter.
Now I would like to extends the configuration by setting up a fifth JBoss instance running the default configuration that needs to serve like a centralized, cluster-wide cache. And this is where difficulties appear. Browsing the Wiki has showed me (due to two articles written by Manik Surtani) that I need to set-up a TreeCache on this fifth machine together with a TcpCacheServer. Each node then synchronizes with the global cache by using the TcpDelegatingCacheLoader. The TreeCache on JBoss instance 5 can then be backed by a JDBCCacheLoader to persist the cached data.
As described in this article, I have installed a default JBoss instance wich I extended with jboss-cache.jar and jgroups.jar in it's default lib folder to provice caching functionality. I'm using JBossCache 1.3.0 together with the JGroups version provided with JBoss 4.0.2 on a JBoss 4.0.2 configuration. In fact, all my JBoss nodes are also running 4.0.2. I configured both the TreeCache and the TcpCacheServer in a HttpCache-service.xml file (located below) and copied this one to the deploy folder together with a mysql-ds.xml file because my TreeCache uses the JDBCCacheLoader.
A first strange thing appeared when starting the AS. I got an error message stating that the datasource was not bounded alltough it is available in the deploy folder. As far as I know, xxx-ds.xml files are always deployed before xxx-service.xml files under normal deployment order configuration. Can anyone explain what the reason for this behaviour might be? Anyway I solved it by using a deploy.last folder that contains my xxx-service.xml file which solved the problem.
A second problem I have is with the TcpCacheServer. As soon as this service is started by the AS, it seems to freeze. The web-console in fact is blocking, so I dont' have the possibility to do anything with the default JBoss configuration. Does anyone have the same behaviour in this circumstances? If so, did anyone found a solution for this one. I know I must be doing something wrong within this set-up but I can't figure out what. Any help would be greatly appreciated off course and any assistance for finding a solution would result in a neat article within the Wiki describing this entire configuration.
Below some of the configuration files I used within my attempt to set-up this configuration.
The JBoss instance serving as cache has the following service definition file called HttpCache-service.xml
<server>
<classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/>
<mbean code="org.jboss.cache.TreeCache" name="jboss.cache:service=HttpCache">
<depends>jboss:service=Naming</depends>
<depends>jboss:service=TransactionManager</depends>
<attribute name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>
<attribute name="NodeLockingScheme">PESSIMISTIC</attribute>
<attribute name="IsolationLevel">REPEATABLE_READ</attribute>
<attribute name="CacheMode">LOCAL</attribute>
<attribute name="UseInterceptorMbeans">TRUE</attribute>
<attribute name="LockAcquisitionTimeout">5000</attribute>
<attribute name="CacheLoaderConfiguration">
<config>
<passivation>TRUE</passivation>
<preload>/</preload>
<shared>FALSE</shared>
<cacheloader>
<class>org.jboss.cache.loader.JDBCCacheLoader</class>
<properties>
cache.jdbc.datasource=java:/HttpCacheDS
</properties>
<fetchPersistentState>true</fetchPersistentState>
<ignoreModifications>false</ignoreModifications>
<async>false</async>
</cacheloader>
</config>
</attribute>
</mbean>
<mbean code="org.jboss.cache.loader.tcp.TcpCacheServer"
name="jboss.cache:service=TcpCacheServer">
<depends optional-attribute-name="Cache" proxy-type="attribute">jboss.cache:service=HttpCache</depends>-->
<attribute name="BindAddress">${jboss.bind.address:localhost}</attribute>
<attribute name="Port">7500</attribute>
<attribute name="MBeanServerName"></attribute>
</mbean>
</server>Each JBoss cluster node (running the all configuration) has a modified tc5-cluster-service.xml file that should (if I understand everything correct) use the global cache to synchronize with:
<server> <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/> <mbean code="org.jboss.cache.TreeCache" name="jboss.cache:service=TomcatClusteringCache"> <depends>jboss:service=Naming</depends> <depends>jboss:service=TransactionManager</depends> <attribute name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute> <attribute name="IsolationLevel">REPEATABLE_READ</attribute> <attribute name="CacheMode">LOCAL</attribute> <attribute name="LockAcquisitionTimeout">10000</attribute> <attribute name="UseReplQueue">false</attribute> <attribute name="UseMarshalling">false</attribute> <attribute name="CacheLoaderConfiguration"> <config> <shared>true</shared> <cacheloader> <class>org.jboss.cache.loader.tcp.TcpDelegatingCacheLoader</class> <properties> host=sinfonix port=7500 </properties> <async>true</async> <fetchPersistentState>false</fetchPersistentState> <ignoreModifications>false</ignoreModifications> </cacheloader> </config> </attribute> </mbean> </server>
My datasource configuration file looks like the following
<datasources> <local-tx-datasource> <jndi-name>HttpCacheDS</jndi-name> <connection-url>jdbc:mysql://localhost:3306/jbosscache</connection-url> <driver-class>com.mysql.jdbc.Driver</driver-class> <user-name>xxxxx</user-name> <password>xxxxx</password> <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter</exception-sorter-class-name> <!-- sql to call when connection is created <new-connection-sql>some arbitrary sql</new-connection-sql> --> <!-- sql to call on an existing pooled connection when it is obtained from pool <check-valid-connection-sql>some arbitrary sql</check-valid-connection-sql> --> <!-- corresponding type-mapping in the standardjbosscmp-jdbc.xml (optional) --> <metadata> <type-mapping>mySQL</type-mapping> </metadata> </local-tx-datasource> </datasources>
Can anyone tell me whether or not this is the appropriate approach to achieve the configuration mentionned above or am I completely wrong and is there another approach that must be followed.
Any reference containing information how to create this kind of set-up would be greatly appreciated. Many thanks in advance to all the people out there, certainly the few who found enough energy to read this question ;-)
Cheers,
Kurt
