0 Replies Latest reply on Aug 20, 2014 1:33 AM by Suhasan Chirravuri

    Infinispan cluster is not stable throws StateTransferCancelledException

    Suhasan Chirravuri Newbie

      Hi

       

      I have configured infinispan for 3 node cluster setup.

      initially from the logs i could see that the 3 nodes are part of cluster but after sometime the each node breaks up from cluster. Throwing the following error

       

      17:17:21,631 INFO  [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,preengsdmbl01-50616) ISPN000094: Received new cluster view: [preengsdmbl01-50616|2] [preengsdmbl01-50616, preengsdmbl02-27644, preengsdmbl03-49041]

      17:20:42,361 ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-4,preengsdmbl01-50616) ISPN000172: Failed to prepare view CacheView{viewId=4, members=[preengsdmbl01-50616, preengsdmbl02-27644]} for cache EVRegion, rolling back to view CacheView{viewId=3, members=[preengsdmbl01-50616, preengsdmbl02-27644, preengsdmbl03-49041]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: org.infinispan.statetransfer.StateTransferCancelledException

        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) [rt.jar:1.6.0_24]

        at java.util.concurrent.FutureTask.get(FutureTask.java:111) [rt.jar:1.6.0_24]

        at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:320) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:250) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:894) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.6.0_24]

        at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.6.0_24]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.6.0_24]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.6.0_24]

        at java.lang.Thread.run(Thread.java:679) [rt.jar:1.6.0_24]

      Caused by: org.infinispan.CacheException: org.infinispan.statetransfer.StateTransferCancelledException

        at org.infinispan.util.Util.rewrapAsCacheException(Util.java:526) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:172) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:489) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.cacheviews.CacheViewsManagerImpl$2.call(CacheViewsManagerImpl.java:304) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.cacheviews.CacheViewsManagerImpl$2.call(CacheViewsManagerImpl.java:301) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        ... 5 more

      Caused by: org.infinispan.statetransfer.StateTransferCancelledException

        at org.infinispan.statetransfer.BaseStateTransferTask.checkIfCancelled(BaseStateTransferTask.java:151) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.statetransfer.DistributedStateTransferTask.doPerformStateTransfer(DistributedStateTransferTask.java:136) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.statetransfer.BaseStateTransferTask.performStateTransfer(BaseStateTransferTask.java:93) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.statetransfer.BaseStateTransferManagerImpl.prepareView(BaseStateTransferManagerImpl.java:334) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.cacheviews.CacheViewsManagerImpl.handlePrepareView(CacheViewsManagerImpl.java:492) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.commands.control.CacheViewControlCommand.perform(CacheViewControlCommand.java:126) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.remoting.InboundInvocationHandlerImpl.handle(InboundInvocationHandlerImpl.java:95) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:226) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:203) [infinispan-core-5.1.8.Final-redhat-1.jar:5.1.8.Final-redhat-1]

        at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:459) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:366) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:602) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.JChannel.up(JChannel.java:716) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.RSVP.up(RSVP.java:172) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.FRAG2.up(FRAG2.java:181) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.pbcast.GMS.up(GMS.java:889) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.UNICAST2.handleDataReceived(UNICAST2.java:759) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.UNICAST2.up(UNICAST2.java:365) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:602) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:177) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:288) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.MERGE2.up(MERGE2.java:205) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.Discovery.up(Discovery.java:359) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.TP.passMessageUp(TP.java:1185) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1733) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1715) [jgroups-3.0.14.Final-redhat-1.jar:3.0.14.Final-redhat-1]

        ... 3 more

       

       

      I am not able to identify the root cause of this issue

       

      Reagards

      Suhasan