Repl_Sync Does not appear to be blocking on transactional wr

[xavierpayne2]

I am using jboss cache 1.2 inside of a JBoss 4.0 cluster (I just replaced the JBoss-Cache.jar).

I have a single external client who is doing remote calls to a stateless session bean inside the appserver which in turn is calling TreeCacheAOP to put a serializable into the cache. In the same thread in my external client I then do a read (which from what I understand should be blocked until the previous write has been replicated to all nodes in the cluster). This read is hitting a different JBoss instance than where the write occured and I end up pulling a null instance of my Serialiable object from the cache... Which would seem to indicate that it has not yet been replicated to that node.

I am using transactions and I have tried both "REAPEATABLE_READ" and "SERIALIZABLE"

I get no exceptions.

Is there a bug in the blocking or is my session bean/reflection stuff interfering?

[ben.wang]

> Is the cache going to be invocable for users via JMX,
> or is the MBean nearly going to be a wrapper that
> registers the embedded cache object with JNDI for
> users.

It will definitely be invokable via JMX. As a matter of fact, if you download the latest from the CVS, generate jboss-cache.jar and put both jboss-cache.jar and cache-service.xml into the ../server/all/deploy directory, then you can already use jmx-console. Note that only attributes are implemented thus far, no methods yet. I'll probably use TransactionalHashtable from JavaGroups as a delegee for the methods. Let's see...

So, yes, access is possible via JMX. But, if you use the cache internally, I'm also thinking of providing a DynamicProxy which implements Cache, so that you have both a strongly typed interface for users of the service and acces to the JMX side.


> Or is a proxy going to be registered to JNDI that
> patches the invocations to the JMX invocation layer?

Yes, that's what I have in mind. Interested ?



Bela

[ben.wang]

What is CacheMode that you are using REPL_SYNC or REPL_ASYNC? If you can try to sleep your client thread for couple seconds before you hit the other instance, just to make sure the replication work first.

-Ben

[xavierpayne2]

I am using REPL_SYNC.

I tried putting in thread sleeps varying from 1 -> 10 seconds and cache still doesn't seem to be getting replicated on writes.

I have also loaded my client in the Eclipse Debugger and verified that the only thread in my client (outside of JBoss) is "Main".

I am sure I could be doing something stupid I just don't know what.

[ben.wang]

Can you post your sample client code then? And maybe the xml config as well.

-Ben

[xavierpayne2]

Here is my config file...

import javax.transaction.UserTransaction;

<mbean code="org.jboss.cache.aop.TreeCacheAop"
 name="jboss.cache:service=TreeCacheAop">


 <depends>jboss:service=Naming</depends>
 <depends>jboss:service=TransactionManager</depends>

 <!--
 Configure the TransactionManager
 -->
 <attribute name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>

 <!--
 Isolation level : SERIALIZABLE
 REPEATABLE_READ (default)
 READ_COMMITTED
 READ_UNCOMMITTED
 NONE
 -->
 <attribute name="IsolationLevel">SERIALIZABLE</attribute>

 <!--
 Valid modes are LOCAL, REPL_ASYNC and REPL_SYNC
 -->
 <attribute name="CacheMode">REPL_SYNC</attribute>

 <!--
 Just used for async repl: use a replication queue
 -->
 <attribute name="UseReplQueue">false</attribute>

 <!--
 Replication interval for replication queue (in ms)
 -->
 <attribute name="ReplQueueInterval">0</attribute>

 <!--
 Max number of elements which trigger replication
 -->
 <attribute name="ReplQueueMaxElements">0</attribute>

 <!-- Name of cluster. Needs to be the same for all clusters, in order
 to find each other
 -->
 <attribute name="ClusterName">TreeCache-Cluster</attribute>

 <!-- JGroups protocol stack properties. Can also be a URL,
 e.g. file:/home/bela/default.xml
 <attribute name="ClusterProperties"></attribute>
 -->

 <attribute name="ClusterConfig">
 <config>
 <!-- UDP: if you have a multihomed machine,
 set the bind_addr attribute to the appropriate NIC IP address -->
 <!-- UDP: On Windows machines, because of the media sense feature
 being broken with multicast (even after disabling media sense)
 set the loopback attribute to true -->
 <UDP mcast_addr="228.1.2.3" mcast_port="48866"
 ip_ttl="64" ip_mcast="true"
 mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
 ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
 loopback="false"/>
 <PING timeout="2000" num_initial_members="3"
 up_thread="false" down_thread="false"/>
 <MERGE2 min_interval="10000" max_interval="20000"/>
 <!-- <FD shun="true" up_thread="true" down_thread="true" />-->
 <FD_SOCK/>
 <VERIFY_SUSPECT timeout="1500"
 up_thread="false" down_thread="false"/>
 <pbcast.NAKACK gc_lag="50" retransmit_timeout="600,1200,2400,4800"
 max_xmit_size="8192" up_thread="false" down_thread="false"/>
 <UNICAST timeout="600,1200,2400" window_size="100" min_threshold="10"
 down_thread="false"/>
 <pbcast.STABLE desired_avg_gossip="20000"
 up_thread="false" down_thread="false"/>
 <FRAG frag_size="8192"
 down_thread="false" up_thread="false"/>
 <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
 shun="true" print_local_addr="true"/>
 <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
 </config>
 </attribute>

 <!--
 Whether or not to fetch state on joining a cluster
 -->
 <attribute name="FetchStateOnStartup">true</attribute>

 <!--
 The max amount of time (in milliseconds) we wait until the
 initial state (ie. the contents of the cache) are retrieved from
 existing members in a clustered environment
 -->
 <attribute name="InitialStateRetrievalTimeout">5000</attribute>

 <!--
 Number of milliseconds to wait until all responses for a
 synchronous call have been received.
 -->
 <attribute name="SyncReplTimeout">10000</attribute>

 <!-- Max number of milliseconds to wait for a lock acquisition -->
 <attribute name="LockAcquisitionTimeout">15000</attribute>


 <!-- Name of the eviction policy class. Not supported now. -->
 <attribute name="EvictionPolicyClass"></attribute>
 </mbean>


 <!-- Uncomment to get a graphical view of the TreeCache MBean above -->
 <mbean code="org.jboss.cache.TreeCacheView" name="jboss.cache:service=TreeCacheView">
 <depends>jboss.cache:service=TreeCache</depends>
 <attribute name="CacheService">jboss.cache:service=TreeCache</attribute>
 </mbean>

Here is a snippet of my write code:

public void addSubscription(String UID, mil.af.rl.jbi.platform.server.util.subscription.impl.Subscription sub)
 {
 System.out.println("-------- " + getClass().getName() + " ------------");
 System.out.println("Adding Subscription to cache:");
 System.out.println("UID: " + UID);
 System.out.println("Subscription: " + sub);

 try
 {
 tx.begin();
 ((Subscriptions)distributedCacheService.get("/SubscriptionService/Subscriptions")).addSubscription(UID, sub);
 tx.commit();
 }
 catch (Exception e)
 {
 try {tx.rollback();}catch(Exception e2){}
 // TODO Auto-generated catch block
 e.printStackTrace();
 }
 }

And a snippet of my read code.

public Subscription getSubscription(String UID)
 {
 Subscription sub = null;

 try{
 tx.begin();
 System.out.println("-------- " + getClass().getName() + " ------------");
 System.out.println("-------- Getting Subscriptions by UID for: ");
 System.out.println("UID: " + UID);

 //Subscription foo = (Subscription) ((Subscriptions)distributedCacheService.getObject("/SubscriptionService/Subscriptions")).getSubscription(UID);

 Subscriptions subs = ((Subscriptions)distributedCacheService.get("/SubscriptionService/Subscriptions"));

 System.out.println("The subscriptions map has these: " + subs.printAllSubscriptions());

 sub = (Subscription) subs.getSubscription(UID);

 System.out.println("The subscription pulled from cache was: " + sub);
 tx.commit();
 }
 catch (Exception e)
 {
 try {tx.rollback();}catch(Exception e2){}
 // TODO Auto-generated catch block
 e.printStackTrace();
 }

 return sub;
 }

distributedCacheService is just a helper class that does all the nasty reflection code to the TreeCacheAop service for me. Both the write and read methods I have provided are from the same JMX Service Class but the write method and the read method end up getting invoked on different servers in the cluster. So the write happens on node 1 and the read happens on node 2 but the client that causes these is single threaded invoking these syncronously via RMI to a session (which in turn invokes these methods).

[xavierpayne2]

To add more detail... The "getSubscription" method that gets invoked on node 2 does return me a Subscriptions object... but its contents (a Hashmap of Subscription objects) are null.

This same exact code does work perfectly if there is only 1 node in the cluster.

[xavierpayne2]

I think I found the answer...

I don't think jboss was reading my jboss-aop.xml file :( So it was just using regular Serialization since my classes do implement Serializable. But I wasn't getting any of the AOP niftyness.

I ran the classes I was stuffing into the TreeCacheAOP through the AopC and deployed those into jboss and everything seems to be fine now. (the blocking until replication occurs as it should).

Now I just need to do some thinking as JBossCache really doesn't seem to like certain hashmap functions (.clone and .values) and my AOP objects have a bunch of Hashmaps in them. :)