Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders

[legolas79]

Hi, thank you for this wonderful product. i have one question, when i start infinispan 8.0.1 on centos, i have random of this error:

12:00:03,925 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.datagrid-infinispan.clustered.xxxxxxxx: org.jboss.msc.service.StartException in service jboss.datagrid-infinispan.clustered.xxxxxxx: Failed to start service

        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1904)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)

Caused by: org.infinispan.commons.CacheException: Unable to invoke method public void org.infinispan.persistence.manager.PersistenceManagerImpl.start() on object of type PersistenceManagerImpl

        at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:172)

        at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)

        at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)

        at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)

        at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)

        at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:218)

        at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:850)

        at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:629)

        at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:580)

        at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:445)

        at org.infinispan.manager.impl.AbstractDelegatingEmbeddedCacheManager.getCache(AbstractDelegatingEmbeddedCacheManager.java:122)

        at org.infinispan.server.infinispan.SecurityActions$5.run(SecurityActions.java:127)

        at org.infinispan.server.infinispan.SecurityActions$5.run(SecurityActions.java:124)

        at org.infinispan.security.Security.doPrivileged(Security.java:76)

        at org.infinispan.server.infinispan.SecurityActions.doPrivileged(SecurityActions.java:60)

        at org.infinispan.server.infinispan.SecurityActions.startCache(SecurityActions.java:132)

        at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:82)

        at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)

        at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)

        ... 3 more

Caused by: org.infinispan.commons.CacheException: Unable to start cache loaders

        at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:173)

        at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:497)

        at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)

        ... 21 more

Caused by: java.util.ConcurrentModificationException

        at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)

        at java.util.ArrayList$Itr.next(ArrayList.java:851)

        at org.infinispan.persistence.factory.CacheStoreFactoryRegistry.createInstance(CacheStoreFactoryRegistry.java:38)

        at org.infinispan.persistence.manager.PersistenceManagerImpl.createLoadersAndWriters(PersistenceManagerImpl.java:540)

        at org.infinispan.persistence.manager.PersistenceManagerImpl.start(PersistenceManagerImpl.java:131)

        ... 25 more

this is the configuration:

<distributed-cache name="xxxxxx" mode="ASYNC" segments="30" owners="1" remote-timeout="20000" >

  <eviction max-entries="12000" strategy="LIRS"  />

  <expiration lifespan="21600000" max-idle="-1" />

  <!--<locking acquire-timeout="30000" concurrency-level="1000" striping="false"/>-->

  <transaction mode="NONE" />

  <file-store

                                                   shared="false" preload="true" passivation="true"

                                                   fetch-state="true"

                                                   read-only="false"

                                                   purge="false"

                                                    path="file-store">

  <write-behind flush-lock-timeout="15000"  modification-queue-size="1024"/>

  </file-store>

  </distributed-cache>

i have 180 cache and every restart I find random errors on different cache. can u help me?

[wdfink]

Did you have only one instance or did you start different instances at the same time which try to rebalance ..

How many items are stored in your FileStore (in the cache before you stop)? Only seen if you have a huge cache?

Is that reproducable with only one cache?

[legolas79]

Hi! thank you for u response. I found a solution.I had to add -Djboss.msc.max.container.threads=1. This can causes problems of performance?

to response at your question :

1) only one instance.

2) no huge cache

3) no i can't reproduce with only one cache.

thank u!