We occasionally see the following error when recycling (restarting) nodes in our Infinispan cluster. All caches are configured with CacheMode.INVALIDATION_ASYNC.

org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl

  at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)

  at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)

  at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)

  at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)

  at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)

  at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)

  at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:812)

  at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)

  at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)

  at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)

...

Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache Instance_6623 on dev-64763

  at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)

  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

  at java.lang.reflect.Method.invoke(Method.java:606)

  at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)

  ... 38 more

Although we have a lot of caches (many 1000's) they are all invalidation caches and so I wouldn't expect there to be much, if any, information to transfer.

I have seen mention of setting <state-transfer timeout="..."/>, however looking at the code that handles configuration it doesn't look like invalidation cache configuration pays attention to <state-transfer />.

I have been seeing this occasionally with 7.0.0.Final. Moving to 7.0.2.Final just to be on the latest.

Wondering if anyone could explain whether or not state-transfer is even relevant here and/or why one might see this in a pure "invalidation" mode.