Cannot get List to replicate across a clustered TreeCacheAop

[bentemperton]

I am currently using JBossCache version 1.2.4 deployed as an mbean in JBoss 4.0.2, running under Java 1.5.

My mbean config file is as follows:

<?xml version="1.0" encoding="UTF-8"?>

<!-- ===================================================================== -->
<!-- -->
<!-- Sample TreeCache Service Configuration -->
<!-- -->
<!-- ===================================================================== -->

<server>

 <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/>


 <!-- ==================================================================== -->
 <!-- Defines TreeCache configuration -->
 <!-- ==================================================================== -->

 <mbean code="org.jboss.cache.aop.TreeCacheAop"
 name="jboss.cache:service=TreeCacheAop">

 <depends>jboss:service=Naming</depends>
 <depends>jboss:service=TransactionManager</depends>

 <!--
 Configure the TransactionManager
 -->
 <attribute name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>

 <!--
 Isolation level : SERIALIZABLE
 REPEATABLE_READ (default)
 READ_COMMITTED
 READ_UNCOMMITTED
 NONE
 -->
 <attribute name="IsolationLevel">REPEATABLE_READ</attribute>

 <!--
 Valid modes are LOCAL, REPL_ASYNC and REPL_SYNC
 -->
 <attribute name="CacheMode">REPL_SYNC</attribute>

 <!--
 Just used for async repl: use a replication queue
 -->
 <attribute name="UseReplQueue">false</attribute>

 <!--
 Replication interval for replication queue (in ms)
 -->
 <attribute name="ReplQueueInterval">0</attribute>

 <!--
 Max number of elements which trigger replication
 -->
 <attribute name="ReplQueueMaxElements">0</attribute>

 <!-- Name of cluster. Needs to be the same for all clusters, in order
 to find each other
 -->
 <attribute name="ClusterName">TreeCache-Cluster</attribute>

 <!-- JGroups protocol stack properties. Can also be a URL,
 e.g. file:/home/bela/default.xml
 <attribute name="ClusterProperties"></attribute>
 -->

 <attribute name="ClusterConfig">
 <!-- Use this for TCPIP -->
 <config>
 <TCP bind_addr="MY_MACHINE" start_port="7800" loopback="true"/>
 <TCPPING initial_hosts="ANOTHER_MACHINE[7800]" port_range="3" timeout="3500"
 num_initial_members="1" up_thread="true" down_thread="true"/>
 <MERGE2 min_interval="5000" max_interval="10000"/>
 <FD shun="true" timeout="2500" max_tries="5" up_thread="true" down_thread="true"/>
 <VERIFY_SUSPECT timeout="1500" down_thread="false" up_thread="false"/>
 <pbcast.NAKACK down_thread="true" up_thread="true" gc_lag="100"
 retransmit_timeout="3000"/>
 <pbcast.STABLE desired_avg_gossip="20000" down_thread="false" up_thread="false"/>
 <pbcast.GMS join_timeout="5000" join_retry_timeout="2000" shun="false"
 print_local_addr="true" down_thread="true" up_thread="true"/>
 <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
 </config>
 </attribute>


 <!--
 Whether or not to fetch state on joining a cluster
 -->
 <attribute name="FetchStateOnStartup">true</attribute>

 <!--
 The max amount of time (in milliseconds) we wait until the
 initial state (ie. the contents of the cache) are retrieved from
 existing members in a clustered environment
 -->
 <attribute name="InitialStateRetrievalTimeout">5000</attribute>

 <!--
 Number of milliseconds to wait until all responses for a
 synchronous call have been received.
 -->
 <attribute name="SyncReplTimeout">10000</attribute>

 <!-- Max number of milliseconds to wait for a lock acquisition -->
 <attribute name="LockAcquisitionTimeout">15000</attribute>

 <!-- Name of the eviction policy class. -->
 <attribute name="EvictionPolicyClass"></attribute>

 <!--
 <attribute name="CacheLoaderClass">org.jboss.cache.loader.bdbje.BdbjeCacheLoader</attribute>
 <attribute name="CacheLoaderConfig">
 location=c:\\tmp\\bdbje
 </attribute>
 <attribute name="CacheLoaderShared">true</attribute>
 <attribute name="CacheLoaderPreload">/</attribute>
 -->

 <!--
 <attribute name="CacheLoaderClass">org.jboss.cache.loader.FileCacheLoader</attribute>
 <attribute name="CacheLoaderConfig">
 location=c:\\tmp
 </attribute>
 <attribute name="CacheLoaderShared">true</attribute>
 <attribute name="CacheLoaderPreload">/</attribute>
 -->


 </mbean>


 <!-- Uncomment to get a graphical view of the TreeCache MBean above -->
 <!-- <mbean code="org.jboss.cache.TreeCacheView" name="jboss.cache:service=TreeCacheView">-->
 <!-- <depends>jboss.cache:service=TreeCacheAop</depends>-->
 <!-- <attribute name="CacheService">jboss.cache:service=TreeCacheAop</attribute>-->
 <!-- </mbean>-->

</server>

I have two servlets - one which creates a list on the cache with one value(InServlet):

 protected void doPost(HttpServletRequest request, HttpServletResponse response)
 throws ServletException, IOException
 {
 try
 {
 MBeanServer server = MBeanServerLocator.locateJBoss();
 TreeCacheMBean cache;
 cache = (TreeCacheMBean) MBeanProxyExt.create(TreeCacheMBean.class, "jboss.cache:service=TreeCacheAop", server);
 List list = new ArrayList(5);
 list.add("Hello");

 cache.put("test.cache.name", "TestKey", list);
 ServletOutputStream out = response.getOutputStream();
 out.println("<HTML>");
 out.println("<BODY>");
 out.println("Added value to cache");
 out.println("</BODY>");
 out.println("</HTML>");


 }
 catch (Exception e)
 {
 e.printStackTrace();
 }
 }

and one which gets the values out of the list on the cache, displays them, then adds one to the list (OutServlet):

protected void doPost(HttpServletRequest request, HttpServletResponse response)
 throws ServletException, IOException
 {
 try
 {
 MBeanServer server = MBeanServerLocator.locateJBoss();
 TreeCacheMBean cache;
 cache = (TreeCacheMBean) MBeanProxyExt.create(TreeCacheMBean.class, "jboss.cache:service=TreeCacheAop", server);

 ServletOutputStream out = response.getOutputStream();
 out.println("<HTML>");
 out.println("<BODY>");
 Object values = cache.get("test.cache.name", "TestKey");
 out.println("The class is: " + values.getClass());
 List listValues = (List)values;
 for (Object o : listValues)
 {
 out.println(" Value in cache is: " + o + "<br/>");
 }
 out.println("</BODY>");
 out.println("</HTML>");

 ((List)cache.get("com.qxlva.test", "TestKey")).add("New Value -" + Calendar.getInstance().getTime());


 }
 catch (Exception e)
 {
 e.printStackTrace();
 }
 }

If I call MY_MACHINE/InServlet then MY_MACHINE/OutServlet, I can see the added values. Calling MY_MACHINE/OutServlet displays another value.

However, calling ANOTHER_MACHINE/OutServlet displays the initial value of the list added to the cache by MY_MACHINE/InServlet, but not the extra values added by MY_MACHINE/OutServlet.

Calling ANOTHER_MACHINE/OutServlet adds items to the cache on ANOTHER_MACHINE, but these are not propagated across to the cache on MY_MACHINE.

If I stop restart JBoss on ANOTHER_MACHINE, then hit ANOTHER_MACHINE/OutServlet, the full of values added from both MY_MACHINE and ANOTHER_MACHINE are shown. This would suggest to me that the list is being propagated across the cluster upon startup, and indeed when the list is added to the cache initially, but not upon value change within the clustered cache.

I have tried wrapping the inserts in both servlets with
cache.getTransactionManager().begin()
{ ...add stuff to cache...}
cache.getTransactionManager().commit()

but it has made no difference.

How can I get the values added to the cache to propagate across the cluster when a value is added to the list?

Many thanks,

Ben

[ben.wang]

Have you tried it first without running it under MBean service? E.g., as a standalone to make sure your enviornment is ok?

-Ben

[brian.stansberry]

OutServlet is not calling cache.put() after adding the new value to the list. A plain TreeCache cannot detect changes made to objects placed in the cache; it will only replicate when put() or remove() are called.

If you want a cache that can automatically detect and replicate changes made to cached objects, you need to use TreeCacheAop. In fact, not having to remember to call put() all the time is one of the key benefits of TreeCacheAop :)

[bentemperton]

Have you tried it first without running it under MBean service? E.g., as a standalone to make sure your enviornment is ok?

-Ben

Yeah, my first attempt was running it as a standalone. I had the same problem, so moved to deploying it as an mbean in an attempt to resolve the situation (unsuccessfully).

[bentemperton]

OutServlet is not calling cache.put() after adding the new value to the list. A plain TreeCache cannot detect changes made to objects placed in the cache; it will only replicate when put() or remove() are called.

If you want a cache that can automatically detect and replicate changes made to cached objects, you need to use TreeCacheAop. In fact, not having to remember to call put() all the time is one of the key benefits of TreeCacheAop :)

I was under the impression I was using TreeCacheAop :). It is defined in my mbean xml config as:

mbean code="org.jboss.cache.aop.TreeCacheAop"

Is this assumption false?

Can you confirm that it's ok to use a list as the value in the cache, and that I don't have to have a wrapper object so that I may "aspectize" the getter and setter of the list?

Many Thanks,

Ben

[brian.stansberry]

Note to self: don't post late at night when should be sleeping :)

Yep, you are using TreeCacheAop. But not the object-based API. Your put should be done using cache.putObject("test.cache.name", list), the get using cache.getObject("test.cache.name").

TreeCacheAop subclasses TreeCache and provides all the superclass behavior plus an extended API. If you use the put/get/remove API inherited from the superclass you get the superclass behavior I described previously.

[brian.stansberry]

Just to clarify -- if you replace your put() calls w/ putObject() and get with getObject(), you won't need to add another putObject() call to OutServlet.

[bentemperton]

Excellent! I made that change and it has worked. You were part right in your initial response, as I was using TreeCacheMBean in my servlets instead of TreeCacheAopMBean :)

Thanks for the help.