Infinispan error as "org.infinispan.commons.CacheException"
rituraj May 27, 2014 7:44 AMwe are running 6 nodes in domain mode where 2 nodes are running on master and 4 are distributed on 2 slaves
master :- node 1 and node 2
slave 1 : - node 3 and node 4
slave 2 : - node 5 and and node 6
there are 2 clusters as
cluster 1: ( node 1 , node 3 and node 5)
cluster 2 : (node 2 , node 4 and node 6)
profile used : HA
we are using modcluster as well
both the clusters have the same war deployed on them and everything is working fine. During my load testing i observed that one of node was stopped responsing(was not able to serve the any incoming request).
Hence I decided to restart it using web console, during restart I m observing the below exception:
06:11:05,670 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 64) MSC000001: Failed to start service jboss.infinispan.web.default-host/testapp: org.jboss.msc.service.StartException in service jboss.infinispan.web.default-host/testapp: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
[Server:node1] at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:91)
[Server:node1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45]
[Server:node1] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45]
[Server:node1] at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45]
[Server:node1] at org.jboss.threads.JBossThread.run(JBossThread.java:122)
[Server:node1] Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
[Server:node1] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
[Server:node1] at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
[Server:node1] at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
[Server:node1] at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
[Server:node1] at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
[Server:node1] at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
[Server:node1] at org.infinispan.CacheImpl.start(CacheImpl.java:675)
[Server:node1] at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:553)
[Server:node1] at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:516)
[Server:node1] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:398)
[Server:node1] at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:412)
[Server:node1] at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:103)
[Server:node1] at org.jboss.as.clustering.infinispan.DefaultCacheContainer.getCache(DefaultCacheContainer.java:94)
[Server:node1] at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)
[Server:node1] at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:86)
[Server:node1] ... 4 more
[Server:node1] Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache default-host/testapp on wildfly-app301p_dc:node1/wildfly-consumer1-cluster
[Server:node1] at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:202)
[Server:node1] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45]
[Server:node1] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45]
[Server:node1] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]
[Server:node1] at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]
[Server:node1] at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:183)
[Server:node1] ... 18 more
[Server:node1]
[Server:node1] 06:11:05,695 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014613: Operation ("add") failed - address: ([("deployment" => "test.war")]) - failure description: {"JBAS014671: Failed services" => {"jboss.infinispan.web.default-host/testapp" => "org.jboss.msc.service.StartException in service jboss.infinispan.web.default-host/testapp: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
[Server:node1] Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
[Server:node1] Caused by: org.infinispan.commons.CacheException: Initial state transfer timed out for cache default-host/testapp on wildfly-app301p_dc:node1/wildfly-consumer1-cluster"}}
[Server:node1] 06:11:05,771 INFO [org.jboss.as.server] (Controller Boot Thread) JBAS018559: Deployed "test.war" (runtime-name : "test.war")
[Server:node1] 06:11:05,772 INFO [org.jboss.as.server] (Controller Boot Thread) JBAS018559: Deployed "property-adapter-module.rar" (runtime-name : "property-adapter-module.rar")
[Server:node1] 06:11:05,772 INFO [org.jboss.as.server] (Controller Boot Thread) JBAS018559: Deployed "mysql-connector-java-5.1.30-bin.jar" (runtime-name : "mysql-connector-java-5.1.30-bin.jar")
[Server:node1] 06:11:05,777 INFO [org.jboss.as.controller] (Controller Boot Thread) JBAS014774: Service status report
[Server:node1] JBAS014777: Services which failed to start: service jboss.infinispan.web.default-host/testapp: org.jboss.msc.service.StartException in service jboss.infinispan.web.default-host/testapp: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete() throws java.lang.InterruptedException on object of type StateTransferManagerImpl
[Server:node1]
[Server:node1] 06:11:05,785 ERROR [org.jboss.as] (Controller Boot Thread) JBAS015875: WildFly 8.0.0.Final "WildFly" started (with errors) in 84750ms - Started 912 of 1044 services (8 services failed or missing dependencies, 192 services are lazy, passive or on-demand)
and after this error we were unable to start the server also it continued to give error like that only ....
can someone please let us know how can we solve the infinispan issue ...?