this is going to be a long post due to code amount, sorry for that :)
So here' s the thing:
last time I checked when I called get() from that 3rd node i got "null", but now things are different. I must have changed something.
This is what I do:
1) I start 2 clasters on local machine and one on the other. I use the following cfg.xml:
<?xml version="1.0" encoding="UTF-8"?>
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:infinispan:config:4.0">
<global>
<transport transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport" clusterName="infinispan-cluster" distributedSyncTimeout="50000">
</transport>
</global>
<default>
<clustering mode="distribution">
<l1 enabled="true" lifespan="10000"/>
<hash numOwners="3"/>
<sync/>
</clustering>
</default>
</infinispan>
2) I try to add 10 items to one of the local clusters and then it suspends and I get error that you can see in this log (local machine, because on the 3rd one there is no new log):
2009-09-04 12:31:19,156 DEBUG [org.infinispan.marshall.jboss.JBossMarshaller] (main) Using JBoss Marshalling based marshaller.
2009-09-04 12:31:19,203 DEBUG [org.infinispan.marshall.jboss.JBossMarshaller] (main) Using JBoss Marshalling based marshaller.
2009-09-04 12:31:19,203 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Starting JGroups Channel
2009-09-04 12:31:19,203 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
2009-09-04 12:31:19,562 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Received new cluster view: [osoz-fd029efaa5-47180|1] [osoz-fd029efaa5-47180, osoz-fd029efaa5-46630]
2009-09-04 12:31:19,609 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Cache local address is osoz-fd029efaa5-46630
2009-09-04 12:31:19,625 DEBUG [org.infinispan.interceptors.InterceptorChain] (main) Interceptor chain size: 6
2009-09-04 12:31:19,625 DEBUG [org.infinispan.interceptors.InterceptorChain] (main) Interceptor chain is:
>> org.infinispan.interceptors.InvocationContextInterceptor
>> org.infinispan.interceptors.DistTxInterceptor
>> org.infinispan.interceptors.NotificationInterceptor
>> org.infinispan.interceptors.DistributionInterceptor
>> org.infinispan.interceptors.DistLockingInterceptor
>> org.infinispan.interceptors.CallInterceptor
2009-09-04 12:31:19,625 TRACE [org.infinispan.interceptors.InvocationContextInterceptor] (main) Invoked with command SizeCommand{containerSize=0} and InvocationContext [NonTxInvocationContext{flags=null}]
2009-09-04 12:31:19,625 TRACE [org.infinispan.interceptors.CallInterceptor] (main) Executing command: SizeCommand{containerSize=0}.
2009-09-04 12:31:51,203 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,osoz-fd029efaa5-46630) Received new cluster view: [osoz-fd029efaa5-47180|2] [osoz-fd029efaa5-47180, o2009-09-04 12:31:51,203 INFO [org.in2009-09-04 12:32:09,468 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,osoz-fd029efaa5-46630) Read version 400
2009-09-04 12:32:09,468 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (Incoming-2,osoz-fd029efaa5-46630) Attempting to execute command: SingleRpcCommand{interceptorChain=null, cacheName='org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME', command=PutMapCommand{map={a9=b9, a0=b0, a1=b1, a2=b2, a3=b3, a4=b4, a5=b5, a6=b6, a7=b7, a8=b8}}} [sender=osoz-fd029efaa5-47180]
2009-09-04 12:32:09,468 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (Incoming-2,osoz-fd029efaa5-46630) Enough waiting; replayIgnored = false, sr STATE_PREEXISTED
2009-09-04 12:32:09,468 TRACE [org.infinispan.commands.remote.BaseRpcCommand] (Incoming-2,osoz-fd029efaa5-46630) Invoking command PutMapCommand{map={a9=b9, a0=b0, a1=b1, a2=b2, a3=b3, a4=b4, a5=b5, a6=b6, a7=b7, a8=b8}}, with originLocal flag set to false.
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.InvocationContextInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Invoked with command PutMapCommand{map={a9=b9, a0=b0, a1=b1, a2=b2, a3=b3, a4=b4, a5=b5, a6=b6, a7=b7, a8=b8}} and InvocationContext [NonTxInvocationContext{flags=null}]
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a9 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a0 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a1 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a2 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a3 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a4 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a5 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a6 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a7 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistributionInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Not doing a remote get for key a8 since entry is mapped to current node, or is in L1
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a9 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a0 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a1 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a2 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a3 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a4 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a5 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a6 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a7 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to lock a8 with acquisition timeout of 10000 millis
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.CallInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Executing command: PutMapCommand{map={a9=b9, a0=b0, a1=b1, a2=b2, a3=b3, a4=b4, a5=b5, a6=b6, a7=b7, a8=b8}}.
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistLockingInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Number of entries in context: 10
2009-09-04 12:32:09,484 TRACE [org.infinispan.container.entries.ReadCommittedEntry] (Incoming-2,osoz-fd029efaa5-46630) Updating entry (key=a8 removed=false valid=true changed=true created=true value=b8]
2009-09-04 12:32:09,484 TRACE [org.infinispan.interceptors.DistLockingInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Releasing lock on [a8] for owner Thread[Incoming-2,osoz-fd029efaa5-46630,5,Thread Pools]
2009-09-04 12:32:09,500 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to unlock a8
2009-09-04 12:32:09,500 TRACE [org.infinispan.container.entries.ReadCommittedEntry] (Incoming-2,osoz-fd029efaa5-46630) Updating entry (key=a7 removed=false valid=true changed=true created=true value=b7]
2009-09-04 12:32:09,500 TRACE [org.infinispan.interceptors.DistLockingInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Releasing lock on [a7] for owner Thread[Incoming-2,osoz-fd029efaa5-46630,5,Thread Pools]
2009-09-04 12:32:09,500 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to unlock a7
2009-09-04 12:32:09,500 TRACE [org.infinispan.container.entries.ReadCommittedEntry] (Incoming-2,osoz-fd029efaa5-46630) Updating entry (key=a6 removed=false valid=true changed=true created=true value=b6]
2009-09-04 12:32:09,500 TRACE [org.infinispan.interceptors.DistLockingInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Releasing lock on [a6] for owner Thread[Incoming-2,osoz-fd029efaa5-46630,5,Thread Pools]
2009-09-04 12:32:09,500 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to unlock a6
2009-09-04 12:32:09,500 TRACE [org.infinispan.container.entries.ReadCommittedEntry] (Incoming-2,osoz-fd029efaa5-46630) Updating entry (key=a5 removed=false valid=true changed=true created=true value=b5]
2009-09-04 12:32:09,500 TRACE [org.infinispan.interceptors.DistLockingInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Releasing lock on [a5] for owner Thread[Incoming-2,osoz-fd029efaa5-46630,5,Thread Pools]
2009-09-04 12:32:09,500 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to unlock a5
2009-09-04 12:32:09,500 TRACE [org.infinispan.container.entries.ReadCommittedEntry] (Incoming-2,osoz-fd029efaa5-46630) Updating entry (key=a4 removed=false valid=true changed=true created=true value=b4]
2009-09-04 12:32:09,500 TRACE [org.infinispan.interceptors.DistLockingInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Releasing lock on [a4] for owner Thread[Incoming-2,osoz-fd029efaa5-46630,5,Thread Pools]
2009-09-04 12:32:09,500 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to unlock a4
2009-09-04 12:32:09,500 TRACE [org.infinispan.container.entries.ReadCommittedEntry] (Incoming-2,osoz-fd029efaa5-46630) Updating entry (key=a3 removed=false valid=true changed=true created=true value=b3]
2009-09-04 12:32:09,500 TRACE [org.infinispan.interceptors.DistLockingInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Releasing lock on [a3] for owner Thread[Incoming-2,osoz-fd029efaa5-46630,5,Thread Pools]
2009-09-04 12:32:09,500 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to unlock a3
2009-09-04 12:32:09,500 TRACE [org.infinispan.container.entries.ReadCommittedEntry] (Incoming-2,osoz-fd029efaa5-46630) Updating entry (key=a2 removed=false valid=true changed=true created=true value=b2]
2009-09-04 12:32:09,500 TRACE [org.infinispan.interceptors.DistLockingInterceptor] (Incoming-2,osoz-fd029efaa5-46630) Releasing lock on [a2] for owner Thread[Incoming-2,osoz-fd029efaa5-46630,5,Thread Pools]
2009-09-04 12:32:09,500 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (Incoming-2,osoz-fd029efaa5-46630) Attempting to unlock a2
2009-09-04 12:32:09,500 TRACE [org.infinispan.container.entries.ReadCommittedEntry] (Incoming-2,osoz-fd029efaa5-46630) Updating entry (key=a1 removed=false valid=true changed=true created=true value=b1]
2009-09-04 12:32:09,500 TRACE [org.infinispan.interceptors.DistLockingInter2009-09-04 12:32:09,531 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (OOB-4,osoz-fd029efaa5-47180) Read version 400
2009-09-04 12:32:24,406 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (main) responses: [sender=osoz-fd029efaa5-46630, retval=SuccessfulResponse, received=true, suspected=false]
[sender=esbopen-9704, retval=null, received=false, suspected=false]
2009-09-04 12:32:24,406 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Cache [osoz-fd029efaa5-47180]: responses for command SingleRpcCommand:
[sender=osoz-fd029efaa5-46630, retval=SuccessfulResponse, received=true, suspected=false]
[sender=esbopen-9704, retval=null, received=false, suspected=false]
2009-09-04 12:32:24,406 TRACE [org.infinispan.remoting.rpc.RpcManagerImpl] (main) replication exception:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for esbopen-9704
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:326)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:87)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:103)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:211)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:197)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:193)
at org.infinispan.interceptors.DistributionInterceptor.handleWriteCommand(DistributionInterceptor.java:265)
at org.infinispan.interceptors.DistributionInterceptor.visitPutMapCommand(DistributionInterceptor.java:148)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitPutMapCommand(AbstractVisitor.java:72)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:183)
at org.infinispan.interceptors.TxInterceptor.visitPutMapCommand(TxInterceptor.java:145)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:48)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:34)
at org.infinispan.commands.AbstractVisitor.visitPutMapCommand(AbstractVisitor.java:72)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:237)
at org.infinispan.CacheDelegate.putAll(CacheDelegate.java:553)
at org.infinispan.CacheDelegate.putAll(CacheDelegate.java:187)
at inf.infini2.main(infini2.java:232)
2009-09-04 12:32:24,421 ERROR [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Replication exception
org.infinispan.util.concurrent.TimeoutException: Replication timeout for esbopen-9704
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:326)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:87)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:103)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:211)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:197)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:193)
at org.infinispan.interceptors.DistributionInterceptor.handleWriteCommand(DistributionInterceptor.java:265)
at org.infinispan.interceptors.DistributionInterceptor.visitPutMapCommand(DistributionInterceptor.java:148)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitPutMapCommand(AbstractVisitor.java:72)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:183)
at org.infinispan.interceptors.TxInterceptor.visitPutMapCommand(TxInterceptor.java:145)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:48)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:34)
at org.infinispan.commands.AbstractVisitor.visitPutMapCommand(AbstractVisitor.java:72)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:237)
at org.infinispan.CacheDelegate.putAll(CacheDelegate.java:553)
at org.infinispan.CacheDelegate.putAll(CacheDelegate.java:187)
at inf.infini2.main(infini2.java:232)
2009-09-04 12:32:24,421 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (main) Execution error:
org.infinispan.util.concurrent.TimeoutException: Replication timeout for esbopen-9704
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:326)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:87)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:103)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:211)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:197)
at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:193)
at org.infinispan.interceptors.DistributionInterceptor.handleWriteCommand(DistributionInterceptor.java:265)
at org.infinispan.interceptors.DistributionInterceptor.visitPutMapCommand(DistributionInterceptor.java:148)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitPutMapCommand(AbstractVisitor.java:72)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:183)
at org.infinispan.interceptors.TxInterceptor.visitPutMapCommand(TxInterceptor.java:145)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:48)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:34)
at org.infinispan.commands.AbstractVisitor.visitPutMapCommand(AbstractVisitor.java:72)
at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:59)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:237)
at org.infinispan.CacheDelegate.putAll(CacheDelegate.java:553)
at org.infinispan.CacheDelegate.putAll(CacheDelegate.java:187)
at inf.infini2.main(infini2.java:232)
So I try to do this in a different way:
1) start three nodes with cfg.xml shown below:
<?xml version="1.0" encoding="UTF-8"?>
<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:infinispan:config:4.0">
<global>
<transport clusterName="demoCluster"/>
</global>
<default>
<clustering mode="replication">
<stateRetrieval timeout="20000" fetchInMemoryState="true"/>
<sync replTimeout="20000"/>
</clustering>
</default>
</infinispan>
And here I've got 2 logs, first is from the local machine:
2009-09-04 12:49:53,218 DEBUG [org.infinispan.marshall.jboss.JBossMarshaller] (main) Using JBoss Marshalling based marshaller.
2009-09-04 12:49:53,265 DEBUG [org.infinispan.marshall.jboss.JBossMarshaller] (main) Using JBoss Marshalling based marshaller.
2009-09-04 12:49:53,265 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Starting JGroups Channel
2009-09-04 12:49:53,265 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
2009-09-04 12:49:53,625 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Received new cluster view: [osoz-fd029efaa5-60337|1] [osoz-fd029efaa5-60337, osoz-fd029efaa5-51501]
2009-09-04 12:49:53,656 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Cache local address is osoz-fd029efaa5-51501
2009-09-04 12:49:53,671 DEBUG [org.infinispan.interceptors.InterceptorChain] (main) Interceptor chain size: 6
2009-09-04 12:49:53,671 DEBUG [org.infinispan.interceptors.InterceptorChain] (main) Interceptor chain is:
>> org.infinispan.interceptors.InvocationContextInterceptor
>> org.infinispan.interceptors.TxInterceptor
>> org.infinispan.interceptors.NotificationInterceptor
>> org.infinispan.interceptors.ReplicationInterceptor
>> org.infinispan.interceptors.LockingInterceptor
>> org.infinispan.interceptors.CallInterceptor
2009-09-04 12:49:53,671 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (main) Data container is 653967
2009-09-04 12:49:53,671 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (main) Initiating state transfer process
2009-09-04 12:49:53,671 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-60337
2009-09-04 12:49:53,687 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,osoz-fd029efaa5-51501) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 12:49:53,687 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,osoz-fd029efaa5-51501) Applying state
2009-09-04 12:49:53,703 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,osoz-fd029efaa5-51501) Read version 400
2009-09-04 12:49:53,703 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,osoz-fd029efaa5-51501) Integrating transaction log
2009-09-04 12:49:53,703 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,osoz-fd029efaa5-51501) Applying commit log
2009-09-04 12:49:53,703 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,osoz-fd029efaa5-51501) Finished applying commit log
2009-09-04 12:49:53,703 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,osoz-fd029efaa5-51501) dests=[osoz-fd029efaa5-60337], command=StateTransferControlCommand{enabled=true}, mode=SYNCHRONOUS, timeout=20000
2009-09-04 12:49:53,718 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,osoz-fd029efaa5-51501) Wrote version 400
2009-09-04 12:49:53,718 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (Incoming-2,osoz-fd029efaa5-51501) real_dests=[osoz-fd029efaa5-60337]
2009-09-04 12:49:53,734 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (OOB-1,osoz-fd029efaa5-51501) Read version 400
2009-09-04 12:49:53,734 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (Incoming-2,osoz-fd029efaa5-51501) responses: [sender=osoz-fd029efaa5-60337, retval=null, received=true, suspected=false]
2009-09-04 12:49:53,734 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,osoz-fd029efaa5-51501) Cache [osoz-fd029efaa5-51501]: responses for command StateTransferControlCommand:
null
2009-09-04 12:49:53,734 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,osoz-fd029efaa5-51501) Retrieving/Applying post-flush commits
2009-09-04 12:49:53,734 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,osoz-fd029efaa5-51501) Applying commit log
2009-09-04 12:49:53,734 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,osoz-fd029efaa5-51501) Finished applying commit log
2009-09-04 12:49:53,734 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,osoz-fd029efaa5-51501) Retrieving/Applying pending prepares
2009-09-04 12:49:52009-09-04 12:49:53,734 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (OOB-3,osoz-fd029efaa5-60337) Read version 400
2009-09-04 12:49:53,734 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (OOB-3,osoz-fd029efaa5-60337) Wrote version 400
2009-09-04 12:49:59,031 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-1,osoz-fd029efaa5-60337) Received new cluster view: [osoz-fd029efaa5-60337|2] [osoz-fd029efaa5-60337, osoz-fd029efaa5-51501, esbopen-3341]
2009-09-04 12:49:59,296 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Received request to generate state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to generate state.
2009-09-04 12:49:59,296 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Generating state. Can provide? true
2009-09-04 12:49:59,296 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Wrote version 400
2009-09-04 12:49:59,296 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Writing 0 StoredEntries to stream
2009-09-04 12:49:59,296 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Transaction log size is 0
2009-09-04 12:49:59,296 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Waiting for a distributed sync block
2009-09-04 12:50:01,203 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Received request to generate state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to generate state.
2009-09-04 12:50:01,203 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Generating state. Can provide? false
2009-09-04 12:50:01,203 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Wrote version 400
2009-09-04 12:50:01,203 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Not providing state!
2009-09-04 12:50:04,031 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Received request to generate state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to generate state.
2009-09-04 12:50:04,031 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Generating state. Can provide? false
2009-09-04 12:50:04,031 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Wrote version 400
2009-09-04 12:50:04,031 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Not providing state!
2009-09-04 12:50:09,156 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Received request to generate state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to generate state.
2009-09-04 12:50:09,156 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Generating state. Can provide? false
2009-09-04 12:50:09,156 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Wrote version 400
2009-09-04 12:50:09,156 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Not providing state!
2009-09-04 12:50:19,109 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Received request to generate state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to generate state.
2009-09-04 12:50:19,109 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Generating state. Can provide? false
2009-09-04 12:50:19,109 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Wrote version 400
2009-09-04 12:50:19,109 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Not providing state!
2009-09-04 12:50:47,093 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-3,osoz-fd029efaa5-60337) Received new cluster view: [osoz-fd029efaa5-60337|3] [osoz-fd029efaa5-60337, osoz-fd029efaa5-51501]
2009-09-04 12:50:59,296 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-60337) Caught while responding to state transfer request
org.infinispan.statetransfer.StateTransferException: java.util.concurrent.TimeoutException: Timed out waiting for a cluster-wide sync to be acquired. (timeout = 60 seconds)
at org.infinispan.statetransfer.StateTransferManagerImpl.generateState(StateTransferManagerImpl.java:164)
at org.infinispan.remoting.InboundInvocationHandlerImpl.generateState(InboundInvocationHandlerImpl.java:78)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.getState(JGroupsTransport.java:456)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:782)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1484)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER$StateProviderHandler.process(STREAMING_STATE_TRANSFER.java:644)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER$StateProviderThreadSpawner$1.run(STREAMING_STATE_TRANSFER.java:573)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.TimeoutException: Timed out waiting for a cluster-wide sync to be acquired. (timeou2009-09-04 12:50:59,703 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (STREAMING_STATE_TRANSFER sender,osoz-fd029efaa5-51501) Caught while responding to state transfer request
org.infinispan.statetransfer.StateTransferException: java.util.concurrent.TimeoutException: Timed out waiting for a cluster-wide sync to be acquired. (timeout = 60 s2009-09-04 12:51:42,015 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Thread-0) Disconnecting and closing JGroups Channel
2009-09-04 12:51:42,078 ERROR [org.jgroups.protocols.UDP] (OOB-7,osoz-fd029efaa5-60337) failed sending message to osoz-fd029efaa5-51501 (93 bytes)
java.lang.Exception: dest=/10.10.8.100:2669 (96 bytes)
at org.jgroups.protocols.UDP._send(UDP.java:208)
at org.jgroups.protocols.UDP.sendUnicast(UDP.java:191)
at org.jgroups.protocols.TP.sendToSingleMember(TP.java:1106)
at org.jgroups.protocols.TP.doSend(TP.java:1090)
at org.jgroups.protocols.TP.send(TP.java:1073)
at org.jgroups.protocols.TP.down(TP.java:934)
at org.jgroups.protocols.Discovery.down(Discovery.java:410)
at org.jgroups.protocols.MERGE2.down(MERGE2.java:161)
at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:334)
at org.jgroups.protocols.FD_ALL.down(FD_ALL.java:220)
at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:76)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:653)
at org.jgroups.protocols.UNICAST.sendAck(UNICAST.java:670)
at org.jgroups.protocols.UNICAST.sendAckForMessage(UNICAST.java:679)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:535)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.p2009-09-04 12:51:42,500 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,osoz-fd029efaa5-51501) Received new cluster view: [osoz-fd029efaa5-51501|4] [osoz-fd029efaa5-51501]
2009-09-04 12:51:42,796 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Thread-0) Disconnecting and closing JGroups Channel
2009-09-04 12:51:42,796 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Thread-0) Stopping the RpcDispatcher
2009-09-04 12:51:42,796 DEBUG [org.infinispan.lifecycle.ComponentStatus] (Thread-0) Ignoring call to stop() as current state is STOPPING
pl.send(Native Method)
at java.net.DatagramSocket.send(Unknown Source)
at org.jgroups.protocols.UDP._send(UDP.java:204)
... 29 more
2009-09-04 12:51:42,390 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Thread-0) Stopping the RpcDispatcher
2009-09-04 12:51:42,390 DEBUG [org.infinispan.lifecycle.ComponentStatus] (Thread-0) Ignoring call to stop() as current state is STOPPING
And the socond, from remote one:
2009-09-04 10:48:56,684 DEBUG [org.infinispan.marshall.jboss.JBossMarshaller] (main) Using JBoss Marshalling based marshaller.
2009-09-04 10:48:56,747 DEBUG [org.infinispan.marshall.jboss.JBossMarshaller] (main) Using JBoss Marshalling based marshaller.
2009-09-04 10:48:56,762 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Starting JGroups Channel
2009-09-04 10:48:56,762 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
2009-09-04 10:48:57,512 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Received new cluster view: [osoz-fd029efaa5-60337|2] [osoz-fd029efaa5-60337, osoz-fd029efaa5-51501, esbopen-3341]
2009-09-04 10:48:57,544 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (main) Cache local address is esbopen-3341
2009-09-04 10:48:57,591 DEBUG [org.infinispan.interceptors.InterceptorChain] (main) Interceptor chain size: 6
2009-09-04 10:48:57,591 DEBUG [org.infinispan.interceptors.InterceptorChain] (main) Interceptor chain is:
>> org.infinispan.interceptors.InvocationContextInterceptor
>> org.infinispan.interceptors.TxInterceptor
>> org.infinispan.interceptors.NotificationInterceptor
>> org.infinispan.interceptors.ReplicationInterceptor
>> org.infinispan.interceptors.LockingInterceptor
>> org.infinispan.interceptors.CallInterceptor
2009-09-04 10:48:57,591 TRACE [org.infinispan.statetransfer.StateTransferManagerImpl] (main) Data container is 22292701
2009-09-04 10:48:57,591 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (main) Initiating state transfer process
2009-09-04 10:48:57,591 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-60337
2009-09-04 10:48:57,591 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:48:57,591 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:48:57,950 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:48:57,950 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Caught while requesting or applying state
org.infinispan.statetransfer.StateTransferException: java.lang.ClassCastException: java.util.HashMap
at org.infinispan.statetransfer.StateTransferManagerImpl.applyInMemoryState(StateTransferManagerImpl.java:333)
at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:304)
at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:74)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:473)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:803)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1437)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:516)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:456)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:226)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:195)
at org.jgroups.protocols.FC.up(FC.java:487)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:915)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:239)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:574)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.protocols.Discovery.up(Discovery.java:235)
at org.jgroups.protocols.PING.up(PING.java:282)
at org.jgroups.protocols.TP.passMessageUp(TP.java:996)
at org.jgroups.protocols.TP.access$100(TP.java:53)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1502)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1479)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.ClassCastException: java.util.HashMap
at org.infinispan.statetransfer.StateTransferManagerImpl.applyInMemoryState(StateTransferManagerImpl.java:328)
... 31 more
2009-09-04 10:48:57,950 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-51501
2009-09-04 10:48:57,950 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:48:57,950 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:48:58,356 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:48:58,356 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Caught while requesting or applying state
org.infinispan.statetransfer.StateTransferException: java.lang.ClassCastException: java.util.HashMap
at org.infinispan.statetransfer.StateTransferManagerImpl.applyInMemoryState(StateTransferManagerImpl.java:333)
at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:304)
at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:74)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:473)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:803)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1437)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:516)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:456)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:226)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:195)
at org.jgroups.protocols.FC.up(FC.java:487)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:915)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:239)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:574)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.protocols.Discovery.up(Discovery.java:235)
at org.jgroups.protocols.PING.up(PING.java:282)
at org.jgroups.protocols.TP.passMessageUp(TP.java:996)
at org.jgroups.protocols.TP.access$100(TP.java:53)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1502)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1479)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.ClassCastException: java.util.HashMap
at org.infinispan.statetransfer.StateTransferManagerImpl.applyInMemoryState(StateTransferManagerImpl.java:328)
... 31 more
2009-09-04 10:48:58,356 WARN [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Could not find available peer for state, backing off and retrying
2009-09-04 10:48:59,559 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-60337
2009-09-04 10:48:59,559 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:48:59,559 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:48:59,669 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:48:59,669 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Provider cannot provide state!
2009-09-04 10:48:59,669 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Caught while requesting or applying state
org.infinispan.statetransfer.StateTransferException: Provider cannot provide state!
at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:313)
at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:74)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:473)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:803)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1437)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:516)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:456)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:226)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:195)
at org.jgroups.protocols.FC.up(FC.java:487)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:915)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:239)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:574)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.protocols.Discovery.up(Discovery.java:235)
at org.jgroups.protocols.PING.up(PING.java:282)
at org.jgroups.protocols.TP.passMessageUp(TP.java:996)
at org.jgroups.protocols.TP.access$100(TP.java:53)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1502)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1479)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
2009-09-04 10:48:59,669 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-51501
2009-09-04 10:48:59,669 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:48:59,669 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:48:59,856 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:48:59,856 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Provider cannot provide state!
2009-09-04 10:48:59,856 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Caught while requesting or applying state
org.infinispan.statetransfer.StateTransferException: Provider cannot provide state!
at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:313)
at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:74)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:473)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:803)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1437)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:516)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:456)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:226)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:195)
at org.jgroups.protocols.FC.up(FC.java:487)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:915)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:239)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:574)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.protocols.Discovery.up(Discovery.java:235)
at org.jgroups.protocols.PING.up(PING.java:282)
at org.jgroups.protocols.TP.passMessageUp(TP.java:996)
at org.jgroups.protocols.TP.access$100(TP.java:53)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1502)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1479)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
2009-09-04 10:48:59,950 WARN [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Could not find available peer for state, backing off and retrying
2009-09-04 10:49:02,356 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-60337
2009-09-04 10:49:02,356 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:49:02,356 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:49:02,481 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:49:02,481 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Provider cannot provide state!
2009-09-04 10:49:02,481 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Caught while requesting or applying state
org.infinispan.statetransfer.StateTransferException: Provider cannot provide state!
at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:313)
at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:74)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:473)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:803)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1437)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:516)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:456)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:226)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:195)
at org.jgroups.protocols.FC.up(FC.java:487)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:915)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:239)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:574)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.protocols.Discovery.up(Discovery.java:235)
at org.jgroups.protocols.PING.up(PING.java:282)
at org.jgroups.protocols.TP.passMessageUp(TP.java:996)
at org.jgroups.protocols.TP.access$100(TP.java:53)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1502)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1479)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
2009-09-04 10:49:02,481 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-51501
2009-09-04 10:49:02,481 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:49:02,481 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:49:02,669 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:49:02,669 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Provider cannot provide state!
2009-09-04 10:49:02,669 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Caught while requesting or applying state
org.infinispan.statetransfer.StateTransferException: Provider cannot provide state!
at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:313)
at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:74)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:473)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:803)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1437)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:516)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:456)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:226)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:195)
at org.jgroups.protocols.FC.up(FC.java:487)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:915)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:239)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:574)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.protocols.Discovery.up(Discovery.java:235)
at org.jgroups.protocols.PING.up(PING.java:282)
at org.jgroups.protocols.TP.passMessageUp(TP.java:996)
at org.jgroups.protocols.TP.access$100(TP.java:53)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1502)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1479)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
2009-09-04 10:49:02,684 WARN [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Could not find available peer for state, backing off and retrying
2009-09-04 10:49:07,497 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-60337
2009-09-04 10:49:07,497 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:49:07,497 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:49:07,606 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:49:07,606 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Provider cannot provide state!
2009-09-04 10:49:07,606 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Caught while requesting or applying state
org.infinispan.statetransfer.StateTransferException: Provider cannot provide state!
at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:313)
at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:74)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:473)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:803)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1437)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:516)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:456)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:226)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:195)
at org.jgroups.protocols.FC.up(FC.java:487)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:915)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:239)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:574)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.protocols.Discovery.up(Discovery.java:235)
at org.jgroups.protocols.PING.up(PING.java:282)
at org.jgroups.protocols.TP.passMessageUp(TP.java:996)
at org.jgroups.protocols.TP.access$100(TP.java:53)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1502)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1479)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
2009-09-04 10:49:07,606 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-51501
2009-09-04 10:49:07,622 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:49:07,622 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:49:07,809 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:49:07,809 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Provider cannot provide state!
2009-09-04 10:49:07,809 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Caught while requesting or applying state
org.infinispan.statetransfer.StateTransferException: Provider cannot provide state!
at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:313)
at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:74)
at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:473)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:803)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:863)
at org.jgroups.JChannel.up(JChannel.java:1437)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:812)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:489)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:516)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:456)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:226)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:195)
at org.jgroups.protocols.FC.up(FC.java:487)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:915)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:239)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:574)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:724)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:143)
at org.jgroups.protocols.FD_ALL.up(FD_ALL.java:205)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:271)
at org.jgroups.stack.Protocol.up(Protocol.java:336)
at org.jgroups.protocols.Discovery.up(Discovery.java:235)
at org.jgroups.protocols.PING.up(PING.java:282)
at org.jgroups.protocols.TP.passMessageUp(TP.java:996)
at org.jgroups.protocols.TP.access$100(TP.java:53)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1502)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1479)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
2009-09-04 10:49:07,809 WARN [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Could not find available peer for state, backing off and retrying
2009-09-04 10:49:17,419 INFO [org.infinispan.remoting.rpc.RpcManagerImpl] (main) Trying to fetch state from osoz-fd029efaa5-60337
2009-09-04 10:49:17,419 TRACE [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,esbopen-3341) Received state for cache named 'org.infinispan.manager.DefaultCacheManager.DEFAULT_CACHE_NAME'. Attempting to apply state.
2009-09-04 10:49:17,419 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Applying state
2009-09-04 10:49:17,575 TRACE [org.infinispan.marshall.VersionAwareMarshaller] (Incoming-2,esbopen-3341) Read version 400
2009-09-04 10:49:17,575 DEBUG [org.infinispan.statetransfer.StateTransferManagerImpl] (Incoming-2,esbopen-3341) Provider cannot provide state!
2009-09-04 10:49:17,575 ERROR