Dead lock: thread can't be interrupted for LockControlCommand
We have a cluster configured in replication mode, and some sample of our code is just like the following:
cache.getAdvancedCache().lock(key);
V v=(V)cache.get(key);
.... modify v...
cache.replace(key,v);
When testing this, and if transactions on different node are trying to lock the same key, it's very likely to have a dead lock. Such dead lock can be detected by dead lock dectecting manager, but it can't interrupt it, so what we get is a TimeoutException. Following is the trace log and exception log, it seems we need set the processing thread for LockControlCommand in DeadlockDetectingInterceptor.
2010-06-27 19:05:24,199 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) Attempting to lock 8001291001234563 with acquisition timeout of 20000 millis
2010-06-27 19:05:24,199 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) Using early dead lock detection
2010-06-27 19:05:25,201 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) Could not acquire lock on '8001291001234563' as it is locked by 'DeadlockDetectingGlobalTransaction{replicatingTo=[], replicationThread=null, coinToss=7874736978665674647} GlobalTransaction:<perf-srm-28667>:7447', check for dead locks
2010-06-27 19:05:25,201 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) Should I interrupt other transaction ? true
2010-06-27 19:05:25,201 TRACE [org.infinispan.transaction.xa.DeadlockDetectingGlobalTransaction] (OOB-11,ana-cluster,perf-srm-28667) Processing thread is null, nothing to interrupt
2010-06-27 19:05:25,201 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) trying to lock for the remaining time: 20000 millis
2010-06-27 19:05:43,195 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (pool-86-thread-1) replication exception:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf-srm2-52217
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:421)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:100)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:124)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:229)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:216)
at org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:199)
at org.infinispan.interceptors.base.BaseRpcInterceptor.visitLockControlCommand(BaseRpcInterceptor.java:59)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.LockingInterceptor.visitLockControlCommand(LockingInterceptor.java:149)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:171)
at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:115)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:48)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:34)
at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:269)
at org.infinispan.CacheDelegate.lock(CacheDelegate.java:305)
at org.infinispan.CacheDelegate.lock(CacheDelegate.java:298)
