-
1. Re: Getting ISPN000172 errors in clustered environment in CR4
dan.berindei Jan 18, 2012 1:20 PM (in response to sandkuma)Sandeep, is this the only error you are seeing in the logs? This one isn't very helpful I'm afraid...
Could you either upload your test code or run the test again with TRACE logging enabled for the org.infinispan category and upload the (zipped) log?
-
2. Re: Getting ISPN000172 errors in clustered environment in CR4
dex80526 Apr 21, 2012 5:41 PM (in response to dan.berindei)We are seeing the similar issue in 5.1.3.Final. I posted exception in other thread.
In our case, later on we see OutOfMemoryError from RpcManagerImpl:
2012-04-20/22:13:53.821/EDT [transport-thread-2] ERROR org.infinispan.remoting.rpc.RpcManagerImpl[156] - ISPN000073: Unexpected error while replicating
java.lang.OutOfMemoryError: Java heap space
What casue this OutOfMemoryError?
The JVM has a big heap space in this case.
Here is the exception we have:
2012-04-20/21:31:49.397/EDT [CacheViewTrigger,node1-x] ERROR org.infinispan.cacheviews.CacheViewInfo[143] - ISPN000164: Rolling back to cache view 54, but last committed view is 57
2012-04-20/21:56:39.586/EDT [CacheViewInstaller-6,node1-x] ERROR org.infinispan.cacheviews.CacheViewsManagerImpl[264] - ISPN000172: Failed to prepare view CacheView{viewId=69, members=[node2-x, node1-x} for cache session, rolling back to view CacheView{viewId=58, members=[node1-x]}
java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:322)
at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:250)
at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:876)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2012-04-20/22:02:37.989/EDT [CacheViewInstaller-6,node1-x] ERROR org.infinispan.cacheviews.CacheViewInfo[143] - ISPN000164: Rolling back to cache view 58, but last committed view is 72
2012-04-20/22:07:49.216/EDT [CacheViewInstaller-9,node1-x] ERROR org.infinispan.cacheviews.CacheViewsManagerImpl[264] - ISPN000172: Failed to prepare view CacheView{viewId=74, members=[node2-x, node1-x]} for cache session, rolling back to view CacheView{viewId=73, members=node1-x]}
java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:258)
at java.util.concurrent.FutureTask.get(FutureTask.java:119)
at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterPrepareView(CacheViewsManagerImpl.java:322)
at org.infinispan.cacheviews.CacheViewsManagerImpl.clusterInstallView(CacheViewsManagerImpl.java:250)
at org.infinispan.cacheviews.CacheViewsManagerImpl$ViewInstallationTask.call(CacheViewsManagerImpl.java:876)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
2012-04-20/22:13:53.821/EDT [transport-thread-2] ERROR org.infinispan.remoting.rpc.RpcManagerImpl[156] - ISPN000073: Unexpected error while replicating
java.lang.OutOfMemoryError: Java heap space
at org.infinispan.io.ExposedByteArrayOutputStream.write(ExposedByteArrayOutputStream.java:91)
at org.jboss.marshalling.OutputStreamByteOutput.write(OutputStreamByteOutput.java:56)
at org.jboss.marshalling.SimpleDataOutput.flush(SimpleDataOutput.java:311)
at org.jboss.marshalling.SimpleDataOutput.finish(SimpleDataOutput.java:353)
at org.jboss.marshalling.AbstractMarshaller.finish(AbstractMarshaller.java:134)
at org.infinispan.marshall.jboss.ExtendedRiverMarshaller.finish(ExtendedRiverMarshaller.java:47)
at org.infinispan.marshall.jboss.AbstractJBossMarshaller.finishObjectOutput(AbstractJBossMarshaller.java:116)
at org.infinispan.marshall.VersionAwareMarshaller.finishObjectOutput(VersionAwareMarshaller.java:
-
3. Re: Getting ISPN000172 errors in clustered environment in CR4
galder.zamarreno Apr 26, 2012 7:04 AM (in response to dex80526)Dex, if you get any OOME, add these options to generate a heap dump:
JAVA_OPTS="$JAVA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/java_heap"
or
JAVA_OPTS="$JAVA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/z/Tmp/java_heap"
We can then take the dump (from dropbox?) and analise it. Thx