1 Reply Latest reply on Jul 15, 2015 4:09 AM by Lowry Cui

    TimeoutException: timeout sending message with Modeshape cluster in EAP 6.3

    Lowry Cui Newbie

      Hi,

      I have a cluster with 2 nodes (Modeshape 3.8.0) and Infinispan is configured with DB persistence (sqlserver), deployed in EAP 6.3 .

      If I use different tables on 2 nodes to store modeshape's repository cache (sample-cache). it's ok.

      If I use same table for store repository cache (sample-cache) for two nodes. I got org.jgroups.TimeoutException: timeout sending message to sha-lri-pc-089/modeshape, when try to start one node after another node started successfully.

       

      17:52:33,979 INFO  [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) JBAS010281: Started repository/system cache from modeshape container

      17:52:51,590 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (ServerService Thread Pool -- 63) ISPN000136: Execution error: org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to sha-lri-pc-089/modeshape

        at org.infinispan.util.Util.rewrapAsCacheException(Util.java:542) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:186) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:515) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:173) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:194) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:251) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:238) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:233) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.rpc.RpcManagerImpl.broadcastRpcCommand(RpcManagerImpl.java:212) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.ReplicationInterceptor.broadcastPrepare(ReplicationInterceptor.java:111) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.ReplicationInterceptor.visitPrepareCommand(ReplicationInterceptor.java:102) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.CacheStoreInterceptor.visitPrepareCommand(CacheStoreInterceptor.java:208) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:111) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:109) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:106) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:115) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.statetransfer.TransactionSynchronizerInterceptor.visitPrepareCommand(TransactionSynchronizerInterceptor.java:61) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:284) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:209) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:113) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:92) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:126) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:175) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:81) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:285) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:334) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:91) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:102) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.modeshape.jcr.txn.Transactions$SimpleTransaction.commit(Transactions.java:296) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.cache.document.WritableSessionCache.save(WritableSessionCache.java:472) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.cache.document.WritableSessionCache.save(WritableSessionCache.java:420) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.SystemContent.save(SystemContent.java:110) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.SystemNamespaceRegistry.register(SystemNamespaceRegistry.java:179) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.JcrRepository$RunningState.<init>(JcrRepository.java:1137) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.JcrRepository$RunningState.<init>(JcrRepository.java:971) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.JcrRepository.doStart(JcrRepository.java:404) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.JcrRepository.login(JcrRepository.java:648) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at org.modeshape.jcr.JcrRepository.login(JcrRepository.java:155) [modeshape-jcr-3.8.0.Final.jar:3.8.0.Final]

        at com.lombardrisk.lrm.commons.resource.JcrRepository.getSession(JcrRepository.java:164) [lrm-config-1.6.0.4.jar:1.6.0.4-r241]

        at com.lombardrisk.lrm.commons.resource.JcrRepository.openRepository(JcrRepository.java:87) [lrm-config-1.6.0.4.jar:1.6.0.4-r241]

        at com.lombardrisk.reform.configuration.ReformConfigurationService.refreshConfiguration(ReformConfigurationService.java:59) [reform-ejb-1.2.1.0.jar:1.2.1.0-r6809]

        at com.lombardrisk.reform.configuration.ReformConfigurationService.start(ReformConfigurationService.java:160) [reform-ejb-1.2.1.0.jar:1.2.1.0-r6809]

        at com.lombardrisk.reform.configuration.ReformConfigurationService.init(ReformConfigurationService.java:41) [reform-ejb-1.2.1.0.jar:1.2.1.0-r6809]

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45]

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45]

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]

        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]

        at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.as.ee.component.ComponentInstantiatorInterceptor.processInvocation(ComponentInstantiatorInterceptor.java:76) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.as.ee.component.ManagedReferenceFieldInjectionInterceptorFactory$ManagedReferenceFieldInjectionInterceptor.processInvocation(ManagedReferenceFieldInjectionInterceptorFactory.java:109) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.as.ee.component.ComponentInstantiatorInterceptor.processInvocation(ComponentInstantiatorInterceptor.java:76) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:272) [jboss-as-ejb3-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:368) [jboss-as-ejb3-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.as.ejb3.tx.LifecycleCMTTxInterceptor.processInvocation(LifecycleCMTTxInterceptor.java:66) [jboss-as-ejb3-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:70) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]

        at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:162) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:135) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:90) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:122) [jboss-as-ejb3-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:137) [jboss-as-ejb3-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:54) [jboss-as-ee-7.4.0.Final-redhat-4.jar:7.4.0.Final-redhat-4]

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45]

        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]

        at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]

        at org.jboss.threads.JBossThread.run(JBossThread.java:122)

      Caused by: org.jgroups.TimeoutException: timeout sending message to sha-lri-pc-089/modeshape

        at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:392)

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:301) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:179) [infinispan-core-5.2.8.CR1.jar:5.2.8.CR1]

        ... 115 more

       

      Has anyone seen this error or have any ideas?

      Thanks in advance.