Memory leak using Infinispan 5.1.8 with Hibernate 4.1.8
dchuiko Feb 15, 2013 11:56 AMHi everybody!
I have encountered a memory leak using Infinispan 5.1.8 for Hibernate's (4.1.8) second level cache on Jboss 7.1.2. The problem is in org.hibernate.cache.infinispan.access.PutFromLoadValidator#pendingPuts cache. For an unknown reason after some time of extensive cache usage it began to contain ImmortalCacheEntry instances. So they could not be purged in org.infinispan.container.DefaultDataContainer#purgeExpired. A breakpoint in ImmortalCacheEntry constructor leaded to the following stack trace:
at org.infinispan.container.entries.ImmortalCacheEntry.<init>(ImmortalCacheEntry.java:50) at org.infinispan.container.InternalEntryFactoryImpl.update(InternalEntryFactoryImpl.java:122) at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:144) at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:198) at org.infinispan.interceptors.locking.ClusteringDependentLogic$AllNodesLogic.commitEntry(ClusteringDependentLogic.java:115) at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:208) at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:287) at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:187) at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:213) at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:147) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitPutKeyValueCommand(NonTransactionalLockingInterceptor.java:68) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:130) at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:132) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:91) at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:62) at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:77) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:345) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1006) at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:716) at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:707) at org.infinispan.CacheSupport.putIfAbsent(CacheSupport.java:78) at org.hibernate.cache.infinispan.access.PutFromLoadValidator.registerPendingPut(PutFromLoadValidator.java:382) at org.hibernate.cache.infinispan.access.TransactionalAccessDelegate.get(TransactionalAccessDelegate.java:74) at org.hibernate.cache.infinispan.collection.TransactionalAccess.get(TransactionalAccess.java:36) at org.hibernate.engine.spi.BatchFetchQueue.isCached(BatchFetchQueue.java:335) at org.hibernate.engine.spi.BatchFetchQueue.getCollectionBatch(BatchFetchQueue.java:312) at org.hibernate.loader.collection.BatchingCollectionInitializer.initialize(BatchingCollectionInitializer.java:72) at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:678) at org.hibernate.event.internal.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:80) at org.hibernate.internal.SessionImpl.initializeCollection(SessionImpl.java:1804) at org.hibernate.collection.internal.AbstractPersistentCollection.forceInitialization(AbstractPersistentCollection.java:681) at org.hibernate.Hibernate.initialize(Hibernate.java:77)
Under debugging it was found that ReadCommittedEntry (down to the stack) has maxIdle = -1 despite the fact that in configuration 60 seconds were set. After that pending-puts cache configuration was changed from
builder.clustering().cacheMode(CacheMode.LOCAL) .transaction().transactionMode(TransactionMode.NON_TRANSACTIONAL) .expiration().maxIdle(TimeUnit.SECONDS.toMillis(60)) .storeAsBinary().enabled(false) .locking().isolationLevel(IsolationLevel.READ_COMMITTED) .jmxStatistics().disable();
to
builder.clustering().cacheMode(CacheMode.LOCAL) .transaction().transactionMode(TransactionMode.NON_TRANSACTIONAL) .expiration().maxIdle(TimeUnit.SECONDS.toMillis(60)).lifespan(TimeUnit.MINUTES.toMillis(6)) .storeAsBinary().enabled(false) .locking().isolationLevel(IsolationLevel.READ_COMMITTED) .jmxStatistics().disable();
and that helped.
Thank you in advance for your help with this problem!