[octopus]

One more problem found. I have 2 nodes in replicated cluster with following descriptor (the same for both nodes):

<?xml version="1.0" encoding="UTF-8"?>

<!-- ===================================================================== -->
<!-- JBoss Cache configuration. -->
<!-- -->
<!-- -->
<!-- ===================================================================== -->

<server>

 <classpath codebase="./lib" archives="jboss-cache.jar, jgroups.jar"/>


 <!-- ==================================================================== -->
 <!-- Defines TreeCache configuration -->
 <!-- ==================================================================== -->

 <mbean code="org.jboss.cache.TreeCache"
 name="jboss.cache:service=TreeCache">

 <depends>jboss:service=Naming</depends>
 <depends>jboss:service=TransactionManager</depends>

<!-- <attribute name="JndiName">java:comp/TreeCache</attribute>-->

 <!--
 Configure the TransactionManager
 -->
 <attribute name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>

 <!--
 Isolation level : SERIALIZABLE
 REPEATABLE_READ (default)
 READ_COMMITTED
 READ_UNCOMMITTED
 NONE
 -->
 <attribute name="IsolationLevel">NONE</attribute>

 <!--
 Valid modes are LOCAL, REPL_ASYNC and REPL_SYNC
 -->
 <attribute name="CacheMode">REPL_ASYNC</attribute>

 <!--
 Just used for async repl: use a replication queue
 -->
 <attribute name="UseReplQueue">false</attribute>

 <!--
 Replication interval for replication queue (in ms)
 -->
 <attribute name="ReplQueueInterval">0</attribute>

 <!--
 Max number of elements which trigger replication
 -->
 <attribute name="ReplQueueMaxElements">0</attribute>

 <!-- Name of cluster. Needs to be the same for all clusters, in order
 to find each other
 -->
 <attribute name="ClusterName">TreeCache-Cluster</attribute>

 <!-- JGroups protocol stack properties. Can also be a URL,
 e.g. file:/home/bela/default.xml
 <attribute name="ClusterProperties"></attribute>
 -->

 <attribute name="ClusterConfig">
 <config>
 <!-- UDP: if you have a multihomed machine,
 set the bind_addr attribute to the appropriate NIC IP address, e.g bind_addr="192.168.0.2"
 -->
 <!-- UDP: On Windows machines, because of the media sense feature
 being broken with multicast (even after disabling media sense)
 set the loopback attribute to true -->
 <UDP mcast_addr="228.1.2.3" mcast_port="45603"
 ip_ttl="64" ip_mcast="true"
 mcast_send_buf_size="150000" mcast_recv_buf_size="80000"
 ucast_send_buf_size="150000" ucast_recv_buf_size="80000"
 loopback="false" bind_addr="192.168.100.155"/>
 <PING timeout="2000" num_initial_members="3"
 up_thread="false" down_thread="false"/>
 <MERGE2 min_interval="10000" max_interval="20000"/>
 <!-- <FD shun="true" up_thread="true" down_thread="true" />-->
 <FD_SOCK/>
 <VERIFY_SUSPECT timeout="1500"
 up_thread="false" down_thread="false"/>
 <pbcast.NAKACK gc_lag="50" retransmit_timeout="600,1200,2400,4800"
 max_xmit_size="8192" up_thread="false" down_thread="false"/>
 <UNICAST timeout="600,1200,2400" window_size="100" min_threshold="10"
 down_thread="false"/>
 <pbcast.STABLE desired_avg_gossip="20000"
 up_thread="false" down_thread="false"/>
 <FRAG frag_size="8192"
 down_thread="false" up_thread="false"/>
 <pbcast.GMS join_timeout="5000" join_retry_timeout="2000"
 shun="true" print_local_addr="true"/>
 <pbcast.STATE_TRANSFER up_thread="true" down_thread="true"/>
 </config>
 </attribute>

 <!--
 Max number of entries in the cache. If this is exceeded, the
 eviction policy will kick some entries out in order to make
 more room
 <attribute name="MaxCapacity">20000</attribute>
 -->

 <!--
 Whether or not to fetch state on joining a cluster
 -->
 <attribute name="FetchStateOnStartup">false</attribute>

 <!--
 The max amount of time (in milliseconds) we wait until the
 initial state (ie. the contents of the cache) are retrieved from
 existing members in a clustered environment
 -->
 <attribute name="InitialStateRetrievalTimeout">5000</attribute>

 <!--
 Number of milliseconds to wait until all responses for a
 synchronous call have been received.
 -->
 <attribute name="SyncReplTimeout">10000</attribute>

 <!-- Max number of milliseconds to wait for a lock acquisition -->
 <attribute name="LockAcquisitionTimeout">15000</attribute>

 <!-- Max number of milliseconds we hold a lock (not currently
 implemented)
 <attribute name="LockLeaseTimeout">60000</attribute>
 -->

 <!-- Name of the eviction policy class. -->
 <attribute name="EvictionPolicyClass">org.jboss.cache.eviction.LRUPolicy</attribute>
 <!-- Specific eviction policy configurations. This is LRU -->
 <attribute name="EvictionPolicyConfig">
 <config>
 <attribute name="wakeUpIntervalSeconds">5</attribute>
 <!-- Cache wide default -->
 <region name="/_default_">
 <attribute name="maxNodes">1000</attribute>
 <attribute name="timeToIdleSeconds">600</attribute>
 </region>


 </mbean>


 <!-- Uncomment to get a graphical view of the TreeCache MBean above -->
 <!--
 <mbean code="org.jboss.cache.TreeCacheView" name="jboss.cache:service=TreeCacheView">
 <depends>jboss.cache:service=TreeCache</depends>
 <attribute name="CacheService">jboss.cache:service=TreeCache</attribute>
 </mbean>
 -->

</server>

One of the nodes keeps throwing exception each 5 sec:

2005-01-24 14:56:02,393 INFO [STDOUT] org.jboss.util.NestedRuntimeException: addWriter(): owner already existed; - nested throwable: (java.lang.IllegalStateException: addWriter(): owner already existed)
2005-01-24 14:56:02,394 INFO [STDOUT] at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:2983)
2005-01-24 14:56:02,394 INFO [STDOUT] at org.jboss.cache.TreeCache.evict(TreeCache.java:1693)
2005-01-24 14:56:02,394 INFO [STDOUT] at org.jboss.cache.eviction.BaseEvictionPolicy.evict(BaseEvictionPolicy.java:31)
2005-01-24 14:56:02,394 INFO [STDOUT] at org.jboss.cache.eviction.LRUAlgorithm.evictCacheNode(LRUAlgorithm.java:198)
2005-01-24 14:56:02,396 INFO [STDOUT] at org.jboss.cache.eviction.LRUAlgorithm.prune(LRUAlgorithm.java:268)
2005-01-24 14:56:02,396 INFO [STDOUT] at org.jboss.cache.eviction.LRUAlgorithm.process(LRUAlgorithm.java:56)
2005-01-24 14:56:02,396 INFO [STDOUT] at org.jboss.cache.eviction.EvictionTimerTask.run(EvictionTimerTask.java:35)
2005-01-24 14:56:02,396 INFO [STDOUT] at java.util.TimerThread.mainLoop(Timer.java:512)
2005-01-24 14:56:02,396 INFO [STDOUT] at java.util.TimerThread.run(Timer.java:462)
2005-01-24 14:56:02,396 INFO [STDOUT] Caused by: java.lang.IllegalStateException: addWriter(): owner already existed
2005-01-24 14:56:02,397 INFO [STDOUT] at org.jboss.cache.lock.LockMap.addWriter(LockMap.java:97)
2005-01-24 14:56:02,397 INFO [STDOUT] at org.jboss.cache.lock.IdentityLock.acquireWriteLock(IdentityLock.java:172)
2005-01-24 14:56:02,400 INFO [STDOUT] at org.jboss.cache.Node.acquireWriteLock(Node.java:424)
2005-01-24 14:56:02,406 INFO [STDOUT] at org.jboss.cache.Node.acquire(Node.java:401)
2005-01-24 14:56:02,406 INFO [STDOUT] at org.jboss.cache.interceptors.LockInterceptor.lock(LockInterceptor.java:199)
2005-01-24 14:56:02,406 INFO [STDOUT] at org.jboss.cache.interceptors.LockInterceptor.invoke(LockInterceptor.java:144)
2005-01-24 14:56:02,406 INFO [STDOUT] at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:40)
2005-01-24 14:56:02,406 INFO [STDOUT] at org.jboss.cache.interceptors.CreateIfNotExistsInterceptor.invoke(CreateIfNotExistsInterceptor.java:50)
2005-01-24 14:56:02,407 INFO [STDOUT] at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:40)
2005-01-24 14:56:02,407 INFO [STDOUT] at org.jboss.cache.interceptors.UnlockInterceptor.invoke(UnlockInterceptor.java:35)
2005-01-24 14:56:02,407 INFO [STDOUT] at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:40)
2005-01-24 14:56:02,408 INFO [STDOUT] at org.jboss.cache.interceptors.ReplicationInterceptor.invoke(ReplicationInterceptor.java:51)
2005-01-24 14:56:02,408 INFO [STDOUT] at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:2978)
2005-01-24 14:56:02,409 INFO [STDOUT] ... 8 more

[ben.wang]

Can you open up a JIRA (jira.jboss.com) issue and supply a unit test case for this? Module is jboss cache and you can assign it to me.

I have tried to reproduce it using replicated/SyncTxUnitTestCase by assigning the islolation level to NONE. But only the case where testConcurrentPut fails. And this should be expected since there is NO locking what so ever so you'd expect concurrency won't work.

-Ben

[octopus]

Can you open up a JIRA (jira.jboss.com) issue and supply a unit test case for this? Module is jboss cache and you can assign it to me.

I have tried to reproduce it using replicated/SyncTxUnitTestCase by assigning the islolation level to NONE. But only the case where testConcurrentPut fails. And this should be expected since there is NO locking what so ever so you'd expect concurrency won't work.

-Ben

Ok I'll try to reproduce it myself and post testcase for it.

Another (minor) problem with new version is "Instance" mbean attribute which prints its value in jmx-console - value in fact is current cache content (TreeCache.toString()) which in my case leaves me no chance to see mbean view page in reasonable time.