2 Replies Latest reply on Mar 26, 2015 11:32 PM by valsaraj007

    SuspectException in clustered environment

    valsaraj007

      Hi,

       

      I have a cluster with 2 nodes named node-00 and node-01. I started both nodes and then stopped node-01. Then started node-01 again and found the following error in node-01 log:

      ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (ServerService Thread Pool -- 70) ISPN000136: Execution error: org.infinispan.remoting.transport.jgroups.SuspectException: Node node-00/ejb was suspected

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:172) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:521) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:281) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.releaseLocksOnFailureBeforePrepare(PessimisticLockingInterceptor.java:284) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPutKeyValueCommand(PessimisticLockingInterceptor.java:113) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:255) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:186) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:206) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:141) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:134) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:79) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:32) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:70) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.CacheImpl.putInternal(CacheImpl.java:878) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.CacheImpl.put(CacheImpl.java:870) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.DecoratedCache.put(DecoratedCache.java:401) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.AbstractDelegatingCache.put(AbstractDelegatingCache.java:276) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.wildfly.clustering.server.registry.RegistryFactoryService$2.invoke(RegistryFactoryService.java:166)

        at org.wildfly.clustering.server.registry.RegistryFactoryService$2.invoke(RegistryFactoryService.java:163)

        at org.jboss.as.clustering.infinispan.invoker.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:34) [wildfly-clustering-infinispan-8.2.0.Final.jar:8.2.0.Final]

        at org.jboss.as.clustering.infinispan.invoker.BatchCacheInvoker.invoke(BatchCacheInvoker.java:48) [wildfly-clustering-infinispan-8.2.0.Final.jar:8.2.0.Final]

        at org.wildfly.clustering.server.registry.RegistryFactoryService.getLocalEntry(RegistryFactoryService.java:170)

        at org.wildfly.clustering.server.registry.RegistryFactoryService.createRegistry(RegistryFactoryService.java:101)

        at org.wildfly.clustering.server.registry.RegistryService.start(RegistryService.java:67)

        at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:86) [wildfly-clustering-common-8.2.0.Final.jar:8.2.0.Final]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_07]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_07]

        at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_07]

        at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.1.1.Final.jar:2.1.1.Final]

      Caused by: SuspectedException

        at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:406)

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:353) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:167) [infinispan-core-6.0.2.Final.jar:6.0.2.Final]

        ... 52 more

       

       

      What may be wrong?

       

      Thanks!