Problems with ReplQueue and RegionBasedMarshalling
Hi all!
I'm having problems with Jboss Cache 1.4 GA (running under JBoss 4.0.4GA with JGroups 2.2.9.3).
The cache is deployed as mbean in JBoss and then it's "fetched" in a war-app. Beacuse of this we are using RegionBasedMarshalling.
The cache is also configured to use a ReplQueue and it looks like this combo is causing trouble. If I disable the ReplQueue it works just fine, and if I disable the RegionBasedMarshalling the ReplQueue works.
The error I get (when using ReplQueueMaxElements=2 and ReplQueueInterval=0) is:
15:42:25,814 ERROR [RpcDispatcher] exception=java.lang.IllegalArgumentException: LegacyTreeCacheMarshaller.extractFqn(): Unknown method call name: _replicate 15:42:25,814 ERROR [ReplicationQueue] failed replicating 2 elements in replication queue java.io.NotSerializableException: RpcDispatcher returned a null. This is most often caused by args for _replicate([_replicate(_put(null, /status, 10.60.15.101:4629, com.hm.website.core.cache.status.SystemStatus@10e0904, true)), _replicate(_put(null, /objects/com/hm/website/domain/navigation/NavigationNode/OBJECTS, working, com.hm.website.domain.navigation.ActionPageNavigationNode@1ba67ec, true))]) not being serializable. at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4166) at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4113) at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4221) at org.jboss.cache.ReplicationQueue.flush(ReplicationQueue.java:149) at org.jboss.cache.ReplicationQueue.add(ReplicationQueue.java:131) at org.jboss.cache.interceptors.BaseRpcInterceptor.putCallOnAsyncReplicationQueue(BaseRpcInterceptor.java:130) at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:98) at org.jboss.cache.interceptors.BaseRpcInterceptor.replicateCall(BaseRpcInterceptor.java:88) at org.jboss.cache.interceptors.ReplicationInterceptor.handleReplicatedMethod(ReplicationInterceptor.java:114) at org.jboss.cache.interceptors.ReplicationInterceptor.invoke(ReplicationInterceptor.java:83) at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:68) at org.jboss.cache.interceptors.TxInterceptor.handleNonTxMethod(TxInterceptor.java:345) at org.jboss.cache.interceptors.TxInterceptor.invoke(TxInterceptor.java:156) at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:68) at org.jboss.cache.interceptors.CacheMgmtInterceptor.invoke(CacheMgmtInterceptor.java:157) at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:5517) at org.jboss.cache.TreeCache.put(TreeCache.java:3677) .....
And this strange error occurs when I use ReplQueueInterval=1000
16:06:56,855 ERROR [RpcDispatcher] exception=java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 16:06:56,886 ERROR [ReplicationQueue] failed replicating 0 elements in replication queue java.io.NotSerializableException: RpcDispatcher returned a null. This is most often caused by args for _replicate([]) not being serializable. at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4166) at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4113) at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:4221) at org.jboss.cache.ReplicationQueue.flush(ReplicationQueue.java:149) at org.jboss.cache.ReplicationQueue$MyTask.run(ReplicationQueue.java:158) at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462)
So, is there a known issue with this combo?
I tried debugging the cache (callRemoteMethods(List mbrs, MethodCall method_call, int mode, boolean exclude_self, long timeout) in TreeCache.java) and it seems like the problem is with the LegacyTreeCacheMarshaller. Should it even be used?
Thanks for any help!
BR, Carl
Here is my init-block in the war-app that connects to the cache in JBoss:
ObjectName objectName;
try {
objectName = new ObjectName(CACHE_MBEAN_OBJECT_NAME);
}
catch (Throwable t) {
throw new NetCacheException("Malformed TreeCache ObjectName");
}
TreeCacheMBean mbean;
try {
mbean = (TreeCacheMBean) MBeanProxyExt.create(TreeCacheMBean.class, objectName);
}
catch (Throwable t) {
log.error("Unable to locate TreeCache MBean under object name " + CACHE_MBEAN_OBJECT_NAME, t);
throw new NetCacheException("Unable to locate TreeCache MBean under object name " + CACHE_MBEAN_OBJECT_NAME);
}
cache = mbean;
log.info("Successfully retrived TreeCache mbean from JBossAS");
try {
cache.registerClassLoader("/", Thread.currentThread().getContextClassLoader());
} catch (RegionNameConflictException e) {
log.error("Could not register classloader!", e);
throw new NetCacheException("Could not register classloader!", e);
}
try {
cache.activateRegion("/");
} catch (RegionNameConflictException e) {
log.error("Could not activateRegion / ", e);
throw new NetCacheException("Could not activateRegion / ", e);
} catch (CacheException e) {
log.error("Could not activateRegion / ", e);
throw new NetCacheException("Could not activateRegion / ", e);
}
And here's my config for the cache:
<?xml version="1.0" encoding="UTF-8"?> <!-- ===================================================================== --> <!-- --> <!-- Customized TreeCache Service Configuration for HM --> <!-- --> <!-- ===================================================================== --> <server> <!-- ==================================================================== --> <!-- Defines TreeCache configuration --> <!-- ==================================================================== --> <mbean code="org.jboss.cache.TreeCache" name="com.hm.website.core.cache:service=HmTreeCache"> <depends>jboss:service=Naming</depends> <depends>jboss:service=TransactionManager</depends> <!-- Configure the TransactionManager --> <attribute name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute> <!-- <attribute name="TransactionManagerLookupClass">org.jboss.cache.DummyTransactionManagerLookup</attribute> --> <!-- Isolation level : SERIALIZABLE REPEATABLE_READ (default) READ_COMMITTED READ_UNCOMMITTED NONE --> <attribute name="IsolationLevel">REPEATABLE_READ</attribute> <!-- Valid modes are LOCAL, REPL_ASYNC and REPL_SYNC --> <attribute name="CacheMode">REPL_ASYNC</attribute> <!-- Just used for async repl: use a replication queue --> <attribute name="UseReplQueue">true</attribute> <!-- Replication interval for replication queue (in ms) --> <attribute name="ReplQueueInterval">1000</attribute> <!-- Max number of elements which trigger replication --> <attribute name="ReplQueueMaxElements">2</attribute> <!-- Name of cluster. Needs to be the same for all clusters, in order to find each other --> <attribute name="ClusterName">hmtreecache-devg3</attribute> <!-- JGroups protocol stack properties. Can also be a URL, e.g. file:/home/bela/default.xml <attribute name="ClusterProperties"></attribute> --> <attribute name="ClusterConfig"> <!-- The default UDP stack: - If you have a multihomed machine, set the UDP protocol's bind_addr attribute to the appropriate NIC IP address, e.g bind_addr="192.168.0.2". - On Windows machines, because of the media sense feature being broken with multicast (even after disabling media sense) set the UDP protocol's loopback attribute to true --> <config> <UDP mcast_addr="230.1.2.17" mcast_port="4557" ip_ttl="32" enable_bundling="true" max_bundle_size="64000" max_bundle_timeout="300" mcast_send_buf_size="10000000" mcast_recv_buf_size="10000000" ucast_send_buf_size="10000000" ucast_recv_buf_size="10000000" loopback="true" use_incoming_packet_handler="false" use_outgoing_packet_handler="true" /> <PING timeout="2000" num_initial_members="3" down_thread="false"/> <MERGE2 min_interval="5000" down_thread="false" max_interval="10000"/> <FD_SOCK down_thread="false" /> <VERIFY_SUSPECT timeout="1500" down_thread="false"/> <pbcast.NAKACK max_xmit_size="60000" down_thread="false" use_mcast_xmit="true" gc_lag="50" retransmit_timeout="300,600,1200,2400,4800" discard_delivered_msgs="true" /> <UNICAST timeout="300,600,1200,2400,3600" down_thread="false"/> <pbcast.STABLE stability_delay="1000" desired_avg_gossip="5000" down_thread="false" max_bytes="250000"/> <pbcast.GMS print_local_addr="true" join_timeout="3000" down_thread="false" join_retry_timeout="2000" shun="true" /> <FC max_credits="1000000" down_thread="false" min_threshold="0.10" /> <FRAG frag_size="60000" down_thread="false" up_thread="true" /> <COMPRESS down_thread="false" min_size="500" compression_level="3" up_thread="true" /> <pbcast.STATE_TRANSFER down_thread="false" up_thread="false" /> </config> </attribute> <!-- Whether or not to fetch state on joining a cluster NOTE this used to be called FetchStateOnStartup and has been renamed to be more descriptive. --> <attribute name="FetchInMemoryState">true</attribute> <!-- The max amount of time (in milliseconds) we wait until the initial state (ie. the contents of the cache) are retrieved from existing members in a clustered environment --> <attribute name="InitialStateRetrievalTimeout">20000</attribute> <!-- Number of milliseconds to wait until all responses for a synchronous call have been received. --> <attribute name="SyncReplTimeout">20000</attribute> <!-- Max number of milliseconds to wait for a lock acquisition --> <attribute name="LockAcquisitionTimeout">15000</attribute> <!-- Indicate whether to use marshalling or not. Set this to true if you are running under a scoped class loader, e.g., inside an application server. Default is "false". --> <!-- Configure Marshalling --> <attribute name="UseRegionBasedMarshalling">true</attribute> <attribute name="InactiveOnStartup">true</attribute> <!-- Name of the eviction policy class. --> <attribute name="EvictionPolicyClass">org.jboss.cache.eviction.LRUPolicy</attribute> <!-- Specific eviction policy configurations. This is LRU --> <attribute name="EvictionPolicyConfig"> <config> <attribute name="wakeUpIntervalSeconds">60</attribute> <!-- Cache wide default --> <region name="/_default_"> <attribute name="maxNodes">0</attribute> <attribute name="timeToLiveSeconds">0</attribute> <attribute name="maxAgeSeconds">0</attribute> </region> <region name="/objects" policyClass="org.jboss.cache.eviction.LRUPolicy"> <attribute name="maxNodes">5000</attribute> <attribute name="maxAgeSeconds">21600</attribute> <attribute name="timeToLiveSeconds">21600</attribute> </region> <region name="/status" policyClass="org.jboss.cache.eviction.LRUPolicy"> <attribute name="maxNodes">0</attribute> <attribute name="timeToLiveSeconds">0</attribute> <attribute name="maxAgeSeconds">0</attribute> </region> </config> </attribute> </mbean> </server>
