1 Reply Latest reply on Jun 30, 2010 4:14 AM by Mircea Markus

    Dead lock: thread can't be interrupted for LockControlCommand

    Chun Li Newbie

      We have a cluster configured in replication mode, and some sample of our code is just like the following:

       

      cache.getAdvancedCache().lock(key);

      V v=(V)cache.get(key);

      .... modify v...

      cache.replace(key,v);

       

      When testing this, and if transactions on different node are trying to lock the same key, it's very likely to have a dead lock. Such dead lock can be detected by dead lock dectecting manager, but it can't interrupt it, so what we get is a TimeoutException. Following is the trace log and exception log, it seems we need set the processing thread for LockControlCommand in DeadlockDetectingInterceptor.

       

       

       

      2010-06-27 19:05:24,199 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) Attempting to lock 8001291001234563 with acquisition timeout of 20000 millis
      2010-06-27 19:05:24,199 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) Using early dead lock detection
      2010-06-27 19:05:25,201 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) Could not acquire lock on '8001291001234563' as it is locked by 'DeadlockDetectingGlobalTransaction{replicatingTo=[], replicationThread=null, coinToss=7874736978665674647} GlobalTransaction:<perf-srm-28667>:7447', check for dead locks
      2010-06-27 19:05:25,201 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) Should I interrupt other transaction ? true
      2010-06-27 19:05:25,201 TRACE [org.infinispan.transaction.xa.DeadlockDetectingGlobalTransaction] (OOB-11,ana-cluster,perf-srm-28667) Processing thread is null, nothing to interrupt
      2010-06-27 19:05:25,201 TRACE [org.infinispan.util.concurrent.locks.DeadlockDetectingLockManager] (OOB-11,ana-cluster,perf-srm-28667) trying to lock for the remaining time: 20000 millis

       

       

       

      2010-06-27 19:05:43,195 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (pool-86-thread-1) replication exception:
      org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf-srm2-52217
              at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:421)
              at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:100)
              at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:124)
              at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:229)
              at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:216)
              at org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:199)
              at org.infinispan.interceptors.base.BaseRpcInterceptor.visitLockControlCommand(BaseRpcInterceptor.java:59)
              at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
              at org.infinispan.interceptors.LockingInterceptor.visitLockControlCommand(LockingInterceptor.java:149)
              at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
              at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
              at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
              at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
              at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
              at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
              at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
              at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
              at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
              at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
              at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:171)
              at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:115)
              at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
              at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:48)
              at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:34)
              at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:147)
              at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:82)
              at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:269)
              at org.infinispan.CacheDelegate.lock(CacheDelegate.java:305)
              at org.infinispan.CacheDelegate.lock(CacheDelegate.java:298)