1 Reply Latest reply on Dec 4, 2014 11:44 AM by Rich Mayfield

    Why do we see state transfer timeout occasionally

    Rich Mayfield Newbie

      We occasionally see the following error when recycling (restarting) nodes in our Infinispan cluster. All caches are configured with CacheMode.INVALIDATION_ASYNC.

       

      org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl

        at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)

        at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)

        at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)

        at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)

        at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)

        at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)

        at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:812)

        at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)

        at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)

        at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)

      ...

      Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache Instance_6623 on dev-64763

        at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:606)

        at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)

        ... 38 more

       

      Although we have a lot of caches (many 1000's) they are all invalidation caches and so I wouldn't expect there to be much, if any, information to transfer.

       

      I have seen mention of setting <state-transfer timeout="..."/>, however looking at the code that handles configuration it doesn't look like invalidation cache configuration pays attention to <state-transfer />.

       

      I have been seeing this occasionally with 7.0.0.Final. Moving to 7.0.2.Final just to be on the latest.

       

      Wondering if anyone could explain whether or not state-transfer is even relevant here and/or why one might see this in a pure "invalidation" mode.