5 Replies Latest reply on Jan 12, 2016 10:42 PM by valsaraj007

    Infinispan TimeoutException with hibernate

    valsaraj007

      I am getting following error in wildfly-8.2.0 log:

      ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (Thread-3467 (HornetQ-client-global-threads-481331925)) ISPN000136: Execution error:

      org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [15 seconds] on key [org.app.entity.User#7673074] for requestor [GlobalTransaction:<app-node-01/hibernate>:852914:local]! Lock held by [GlobalTransaction:<app-node-01/hibernate>:853192:local]

        at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)

        at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)

        at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)

        at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:98)

        at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitClearCommand(PessimisticLockingInterceptor.java:194)

        at org.infinispan.commands.write.ClearCommand.acceptVisitor(ClearCommand.java:38)

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

        at org.infinispan.commands.AbstractVisitor.visitClearCommand(AbstractVisitor.java:47)

        at org.infinispan.commands.write.ClearCommand.acceptVisitor(ClearCommand.java:38)

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

        at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:255)

        at org.infinispan.interceptors.TxInterceptor.visitClearCommand(TxInterceptor.java:206)

        at org.infinispan.commands.write.ClearCommand.acceptVisitor(ClearCommand.java:38)

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

        at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

        at org.infinispan.commands.AbstractVisitor.visitClearCommand(AbstractVisitor.java:47)

        at org.infinispan.commands.write.ClearCommand.acceptVisitor(ClearCommand.java:38)

        at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

        at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

        at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

        at org.infinispan.commands.AbstractVisitor.visitClearCommand(AbstractVisitor.java:47)

        at org.infinispan.commands.write.ClearCommand.acceptVisitor(ClearCommand.java:38)

        at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

        at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306)

        at org.infinispan.CacheImpl.clearInternal(CacheImpl.java:443)

        at org.infinispan.CacheImpl.clear(CacheImpl.java:438)

        at org.infinispan.CacheImpl.clear(CacheImpl.java:433)

        at org.infinispan.AbstractDelegatingCache.clear(AbstractDelegatingCache.java:291)

        at org.hibernate.cache.infinispan.access.TransactionalAccessDelegate.removeAll(TransactionalAccessDelegate.java:223) [hibernate-infinispan-4.3.7.Final.jar:4.3.7.Final]

        at org.hibernate.cache.infinispan.entity.TransactionalAccess.removeAll(TransactionalAccess.java:84) [hibernate-infinispan-4.3.7.Final.jar:4.3.7.Final]

       

      What will be the impact of this? Which section and attributes we need to modify to fix this? Please advice.

        • 1. Re: Infinispan TimeoutException with hibernate
          valsaraj007

          After adjusting the values I found, it is invalidation cache timeout. I was running with single node. So what may be the scenario that causes invalidation and timeout? Is there any alternative to avoid infinispan related timeouts?

          • 2. Re: Infinispan TimeoutException with hibernate
            pferraro

            It looks to me like your application is calling Cache.evictAll() - is there a specific reason for this?  An evictAll needs to acquire locks on all entities in the cache, and is thus an expensive operation.  I strongly recommend against doing this too aggressively in WF8.  In WF9 and 10, this is a lock-free operation.

            • 3. Re: Infinispan TimeoutException with hibernate
              valsaraj007

              Hi Paul pferraro,

              We are running in full ha mode with <distributable /> enabled. There is limited RAM allocated on each node to check results. We have some entities @Cacheable in our code. But there is no code written explicitly to call evictAll(). Can you tell me what are the situations this method will be called if we don't call it explicitly?

               

              Is there any way to avoid this?

               

              Thank you!

              • 4. Re: Infinispan TimeoutException with hibernate
                pferraro

                It looks like evictAll() is triggered (among other things) as a result of bulk UPDATE/DELETE HQL statements.  In general, I wouldn't use @Cacheable for entities on which you intend to perform bulk operations - *especially* if you have limited RAM allocated to these nodes.

                1 of 1 people found this helpful
                • 5. Re: Infinispan TimeoutException with hibernate
                  valsaraj007

                  Hi Paul pferraro,

                  We don't have bulk row update/delete while using application from UI. It's only for scheduled jobs and those are not executed. This exception coming when we run load test by simulating 50/100 simultaneos users. There is single row updates for each users on same table, same row which keeps generating unique id for each user and submitting new rows in other table by setting data and unique id. Will this cause any issue?


                  Thanks for help!