Huge burst of java.lang.IllegalArgumentException: Segments [18] are not owned by <slave-node>:<server-name>
thiago.presa Feb 4, 2016 3:41 PMWe have a Wildfly 10 CR4 cluster in domain mode with 2 slaves, and we're getting a huge burst of the following log entry:
10:57:31,065 WARN [org.infinispan.statetransfer.StateConsumerImpl] (transport-thread--p13-t16) ISPN000209: Failed to retrieve transactions for segments [18] of cache dist from node <slave-host>:<server-name>: org.infinispan.remoting.RemoteException: ISPN000217: Received exception from <slave-host>:<server-name>, see cause for remote stack trace
at org.infinispan.remoting.transport.AbstractTransport.checkResponse(AbstractTransport.java:44)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:750)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$80(JGroupsTransport.java:589)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
at org.infinispan.remoting.transport.jgroups.SingleResponseFuture.futureDone(SingleResponseFuture.java:30)
at org.jgroups.blocks.Request.checkCompletion(Request.java:169)
at org.jgroups.blocks.UnicastRequest.receiveResponse(UnicastRequest.java:83)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:398)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:250)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:684)
at org.jgroups.JChannel.up(JChannel.java:738)
at org.jgroups.fork.ForkProtocolStack.up(ForkProtocolStack.java:119)
at org.jgroups.stack.Protocol.up(Protocol.java:374)
at org.jgroups.protocols.FORK.up(FORK.java:114)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:165)
at org.jgroups.protocols.FlowControl.up(FlowControl.java:394)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:1045)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST3.deliverMessage(UNICAST3.java:1064)
at org.jgroups.protocols.UNICAST3.handleDataReceived(UNICAST3.java:779)
at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:426)
at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:652)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:155)
at org.jgroups.protocols.FD.up(FD.java:260)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:310)
at org.jgroups.protocols.MERGE3.up(MERGE3.java:285)
at org.jgroups.protocols.Discovery.up(Discovery.java:295)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1577)
at org.jgroups.protocols.TP$MyHandler.run(TP.java:1796)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Segments [18] are not owned by <slave-host>:<server-name>
at org.infinispan.statetransfer.StateProviderImpl.getTransactionsForSegments(StateProviderImpl.java:166)
at org.infinispan.statetransfer.StateRequestCommand.perform(StateRequestCommand.java:67)
at org.infinispan.remoting.inboundhandler.BasePerCacheInboundInvocationHandler.invokePerform(BasePerCacheInboundInvocationHandler.java:92)
at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:34)
... 3 more
It's funny because server-name is not the name of the server where this log entry is found. What may be causing this?