1 2 Previous Next 20 Replies Latest reply on Sep 25, 2016 4:39 PM by jamat

    HTTP Session Replication - org.infinispan.util.concurrent.TimeoutException

    Allen Gilbert Newbie

      Hello,

       

      We recently upgraded our two-server cluster from JBoss EAP 6.1.0 Alpha to Wildfly 8.2.0.Final. We run a web app built with Wicket (http://wicket.apache.org/), and use Wildfly’s distributed web cache for http session failover. Ever since the upgrade, we’ve been receiving intermittent org.infinispan.util.concurrent.TimeoutExceptions (sample stack trace below).

       

      The exception occurs when a client sends multiple requests to our application (e.g. in different browser tabs), and one page (or AJAX action) takes longer than 15 seconds to render/complete (way too long for a user to wait, but that’s a different problem). The long-running request causes the user’s other requests to block (ostensibly because of an Infinispan lock on the session), and they eventually fail with the org.infinispan.util.concurrent.TimeoutException.

       

      Here’s our distributed web cache config from standalone-full-ha.xml:

       

        <cache-container name="web" default-cache="dist" module="org.wildfly.clustering.web.infinispan">
           <transport lock-timeout="60000"/>
           <distributed-cache name="dist" mode="ASYNC" batching="true" l1-lifespan="0" owners="2">
                <file-store/>
           </distributed-cache>
        </cache-container>
      
      

       

      We experimented with other locking isolation levels (e.g. READ_UNCOMMITTED), but that didn’t make any difference. We also tried turning off batching, but that caused Wildfly to complain about our cache config on startup. So, we’re wondering: is there a way to configure Infinispan with a distributed web cache that doesn’t effectively queue requests from the same user (i.e. session)? Concurrent session access seemed to work well in JBoss EAP 6.1.0.

       

      2015-03-30 11:26:25,240 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-8) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [15 seconds] on key [4KPG9Zr7y8hvxHMlOLNHWgVs] for requestor [GlobalTransaction:<null>:15:local]! Lock held by [GlobalTransaction:<null>:14:local]
      at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)
      at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)
      at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)
      at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitGetKeyValueCommand(PessimisticLockingInterceptor.java:70)
      at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
      at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)
      at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:74)
      at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
      at org.infinispan.interceptors.TxInterceptor.enlistReadAndInvokeNext(TxInterceptor.java:226)
      at org.infinispan.interceptors.TxInterceptor.visitGetKeyValueCommand(TxInterceptor.java:221)
      at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
      at org.infinispan.interceptors.CacheMgmtInterceptor.visitGetKeyValueCommand(CacheMgmtInterceptor.java:92)
      at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
      at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)
      at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)
      at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:74)
      at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)
      at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:79)
      at org.infinispan.commands.AbstractVisitor.visitGetKeyValueCommand(AbstractVisitor.java:74)
      at org.infinispan.commands.read.GetKeyValueCommand.acceptVisitor(GetKeyValueCommand.java:40)
      at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)
      at org.infinispan.CacheImpl.get(CacheImpl.java:377)
      at org.infinispan.DecoratedCache.get(DecoratedCache.java:396)
      at org.infinispan.AbstractDelegatingCache.get(AbstractDelegatingCache.java:271)
      at org.jboss.as.clustering.infinispan.invoker.Locator$FindOperation.invoke(Locator.java:54)
      at org.jboss.as.clustering.infinispan.invoker.Locator$LockingFindOperation.invoke(Locator.java:71)
      at org.jboss.as.clustering.infinispan.invoker.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:34)
      at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:87)
      at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:109)
      at org.wildfly.clustering.web.infinispan.session.coarse.CoarseSessionFactory.findValue(CoarseSessionFactory.java:55)
      at org.wildfly.clustering.web.infinispan.session.InfinispanSessionManager.findSession(InfinispanSessionManager.java:148)
      at org.wildfly.clustering.web.undertow.session.DistributableSessionManager.getSession(DistributableSessionManager.java:115)
      at io.undertow.servlet.spec.ServletContextImpl.getSession(ServletContextImpl.java:688) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]
      at io.undertow.servlet.spec.HttpServletRequestImpl.getSession(HttpServletRequestImpl.java:364) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]
      at org.jboss.weld.servlet.SessionHolder.requestInitialized(SessionHolder.java:47) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]
      at org.jboss.weld.servlet.HttpContextLifecycle.requestInitialized(HttpContextLifecycle.java:212) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]
      at org.jboss.weld.servlet.WeldInitialListener.requestInitialized(WeldInitialListener.java:160) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]
      at io.undertow.servlet.core.ApplicationListeners.requestInitialized(ApplicationListeners.java:216) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]
      at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:260) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]
      at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:247) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]
      at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:76) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]
      at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:166) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]
      at io.undertow.server.Connectors.executeRootHandler(Connectors.java:197) [undertow-core-1.1.0.Final.jar:1.1.0.Final]
      at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:759) [undertow-core-1.1.0.Final.jar:1.1.0.Final]
      
      
        1 2 Previous Next