1 2 Previous Next 20 Replies Latest reply on Mar 14, 2016 11:28 AM by arnab_ghosh

    Timeout Exception when using EJB Infinispan cache

    arnab_ghosh

      Hi All,

       

      I was previously using default simple cache for Stateful SessionBean caching. But after having memory issues (which shows a large amount of session bean objects in heap) and no documentation, I switched to using "distributable" cache already defined in standalone-full.xml. The cache configuration look like below:

       

      <session-bean> 
        <stateful default-access-timeout="5000" cache-ref="distributable"/> 
      </session-bean> 
      <caches> 
        <cache name="distributable" passivation-store-ref="infinispan" aliases="passivating clustered"/> 
      </caches> 
      <passivation-stores> 
        <passivation-store name="infinispan" cache-container="ejb" max-size="10000"/> 
      </passivation-stores> 
      
      
      <subsystem xmlns="urn:jboss:domain:infinispan:2.0"> 
        <cache-container name="ejb" default-cache="passivation" module="org.wildfly.clustering.ejb.infinispan" aliases="sfsb" statistics-enabled="true"> 
        <local-cache name="passivation" batching="true" statistics-enabled="true" > 
        <file-store passivation="true" purge="true" /> 
        <eviction strategy="LRU" max-entries="10000"/> 
        <expiration max-idle="600000" lifespan="900000"/> 
        </local-cache> 
        <local-cache name="persistent" batching="true"> 
        <file-store passivation="false" purge="false"/> 
        </local-cache> 
        </cache-container> 
      </subsystem> 
      
      

       

      Now when application usage load increased I am seeing below exceptions:

       

      2016-02-02 15:53:36,552 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-3) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to acq

      uire lock after [15 seconds] on key [a0863cae-1b01-4432-98bd-0cfa15c28dc7] for requestor [GlobalTransaction:<null>:24027:local]! Lock held by [GlobalTransaction:<null>:23648:local]

              at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)

              at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)

              at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)

              at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:291)

              at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitReplaceCommand(PessimisticLockingInterceptor.java:179)

              at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

              at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

              at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:255)

              at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:201)

              at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)

              at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:139)

              at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

              at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

              at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

              at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

              at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

              at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

              at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

              at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306)

              at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:943)

              at org.infinispan.CacheImpl.replace(CacheImpl.java:936)

              at org.infinispan.DecoratedCache.replace(DecoratedCache.java:451)

              at org.infinispan.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:246)

              at org.jboss.as.clustering.infinispan.invoker.Mutator$MutateOperation.invoke(Mutator.java:50)

              at org.jboss.as.clustering.infinispan.invoker.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:34)

              at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:87)

              at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroupFactory$BeanGroupMutator.mutate(InfinispanBeanGroupFactory.java:109)

              at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroup.close(InfinispanBeanGroup.java:120)

              at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager.passivated(InfinispanBeanManager.java:249)

              at sun.reflect.GeneratedMethodAccessor125.invoke(Unknown Source) [:1.8.0_66]

              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_66]

              at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_66]

              at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:207)

              at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)

              at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:229)

              at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:192)

              at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryPassivated(CacheNotifierImpl.java:374)

              at org.infinispan.eviction.PassivationManagerImpl.passivate(PassivationManagerImpl.java:70)

              at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:243)

              at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:234)

              at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.remove(BoundedConcurrentHashMap.java:1533)

              at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.removeEldestEntry(BoundedConcurrentHashMap.java:528)

              at java.util.LinkedHashMap.afterNodeInsertion(LinkedHashMap.java:299) [rt.jar:1.8.0_66]

              at java.util.HashMap.putVal(HashMap.java:663) [rt.jar:1.8.0_66]

              at java.util.HashMap.put(HashMap.java:611) [rt.jar:1.8.0_66]

       

      at java.util.HashMap.put(HashMap.java:611) [rt.jar:1.8.0_66]

              at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.onEntryMiss(BoundedConcurrentHashMap.java:469)

              at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.put(BoundedConcurrentHashMap.java:1427)

              at org.infinispan.util.concurrent.BoundedConcurrentHashMap.put(BoundedConcurrentHashMap.java:2025)

              at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:170)

              at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:163)

              at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitCacheEntry(ClusteringDependentLogic.java:203)

              at org.infinispan.interceptors.locking.ClusteringDependentLogic$LocalLogic.commitEntry(ClusteringDependentLogic.java:277)

              at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:317)

              at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:542)

              at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:303)

              at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:98)

              at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:78)

              at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:83)

              at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36)

              at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:114)

              at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:101)

              at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

              at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

              at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

              at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

              at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

              at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

              at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

              at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

              at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

              at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

              at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

              at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:154)

              at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58)

              at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:263)

              at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:312)

              at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:69)

              at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80)

              at org.wildfly.clustering.ejb.infinispan.InfinispanBatcher$NewTransactionBatch.close(InfinispanBatcher.java:149)

              at org.jboss.as.ejb3.cache.distributable.DistributableCache.create(DistributableCache.java:109) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

              at org.jboss.as.ejb3.component.stateful.StatefulComponentSessionIdGeneratingInterceptor.processInvocation(StatefulComponentSessionIdGeneratingInterceptor.java:57) [wildfly-ejb3-8.2.0.Final.

      jar:8.2.0.Final]

              at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

              at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

              at org.jboss.as.ee.component.ViewService$DefaultViewInstanceFactory.createViewInstance(ViewService.java:292)

              at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:177)

       

      Please note our system is not distributed and any HA configuration is not enabled.

      Any work around ?

       

      Regards

      Arnab

        • 1. Re: Timeout Exception when using EJB Infinispan cache
          arnab_ghosh

          pferraro can you please take a look at this one ? For now I increased acquire timeout

           

              <locking striping="false" acquire-timeout="120000"/>
          

           

          Not sure, if this is the correct thing to do.

           

          Regards

          Arnab

          • 2. Re: Timeout Exception when using EJB Infinispan cache
            arnab_ghosh

            The issue is not resolved. Here is exception trace:

             

            2016-02-08 14:48:55,093 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-22) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to ac

            quire lock after [120 seconds] on key [122c0375-f913-4acb-8502-a53c68f580e3] for requestor [GlobalTransaction:<null>:49629:local]! Lock held by [GlobalTransaction:<null>:48976:local]

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:291)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitReplaceCommand(PessimisticLockingInterceptor.java:179)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:255)

                    at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:201)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:139)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306)

                    at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:943)

                    at org.infinispan.CacheImpl.replace(CacheImpl.java:936)

                    at org.infinispan.DecoratedCache.replace(DecoratedCache.java:451)

                    at org.infinispan.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:246)

                    at org.jboss.as.clustering.infinispan.invoker.Mutator$MutateOperation.invoke(Mutator.java:50)

                    at org.jboss.as.clustering.infinispan.invoker.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:34)

                    at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:87)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroupFactory$BeanGroupMutator.mutate(InfinispanBeanGroupFactory.java:109)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroup.close(InfinispanBeanGroup.java:120)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager.passivated(InfinispanBeanManager.java:249)

                    at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source) [:1.8.0_66]

                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_66]

                    at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_66]

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:207)

                    at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:229)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:192)

                    at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryPassivated(CacheNotifierImpl.java:374)

                    at org.infinispan.eviction.PassivationManagerImpl.passivate(PassivationManagerImpl.java:70)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:243)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:234)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.remove(BoundedConcurrentHashMap.java:1533)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.removeEldestEntry(BoundedConcurrentHashMap.java:528)

                    at java.util.LinkedHashMap.afterNodeInsertion(LinkedHashMap.java:299) [rt.jar:1.8.0_66]

                    at java.util.HashMap.putVal(HashMap.java:663) [rt.jar:1.8.0_66]

                    at java.util.HashMap.put(HashMap.java:611) [rt.jar:1.8.0_66]

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.onEntryMiss(BoundedConcurrentHashMap.java:469)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.put(BoundedConcurrentHashMap.java:1427)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap.put(BoundedConcurrentHashMap.java:2025)

                    at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:170)

                    at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:163)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitCacheEntry(ClusteringDependentLogic.java:203)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$LocalLogic.commitEntry(ClusteringDependentLogic.java:277)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:317)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:542)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:303)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:98)

            at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:78)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:83)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:114)

                    at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:101)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:154)

                    at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58)

                    at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:263)

                    at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:312)

                    at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:69)

                    at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBatcher$NewTransactionBatch.close(InfinispanBatcher.java:149)

                    at org.jboss.as.ejb3.cache.distributable.DistributableCache.create(DistributableCache.java:109) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.as.ejb3.component.stateful.StatefulComponentSessionIdGeneratingInterceptor.processInvocation(StatefulComponentSessionIdGeneratingInterceptor.java:57) [wildfly-ejb3-8.2.0.Final.

            jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

                    at org.jboss.as.ee.component.ViewService$DefaultViewInstanceFactory.createViewInstance(ViewService.java:292)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:177)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:173)

                    at org.jboss.as.ee.component.ViewManagedReferenceFactory.getReference(ViewManagedReferenceFactory.java:56)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:131)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:81)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:202)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)

                    at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:235)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:188)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacadeWildfly(AOMAServiceSessionBeanFactory.java:213) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacade(AOMAServiceSessionBeanFactory.java:183) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction$SimpleAction.loadSessionBean(SimpleDispatcherAction.java:555) [aoma.jar:]

                    at com.sonymusic.aoma.qc.notes.QCNotesVideoSaveAction.doAction(QCNotesVideoSaveAction.java:52) [aoma.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction.executeAction(SimpleDispatcherAction.java:432) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet$DispatcherAction.execute(AOMADispatcherServlet.java:267) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet.service(AOMADispatcherServlet.java:80) [aoma.jar:]

                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

                    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:63) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final

              at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:247) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:76) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:166) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:197) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:759) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_66]

                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_66]

                    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_66]

             

             

            2016-02-08 14:50:55,135 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-22) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to ac

            quire lock after [120 seconds] on key [122c0375-f913-4acb-8502-a53c68f580e3] for requestor [GlobalTransaction:<null>:50750:local]! Lock held by [GlobalTransaction:<null>:48976:local]

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:291)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitReplaceCommand(PessimisticLockingInterceptor.java:179)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:255)

                    at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:201)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:139)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306)

                    at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:943)

                    at org.infinispan.CacheImpl.replace(CacheImpl.java:936)

                    at org.infinispan.DecoratedCache.replace(DecoratedCache.java:451)

                    at org.infinispan.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:246)

                    at org.jboss.as.clustering.infinispan.invoker.Mutator$MutateOperation.invoke(Mutator.java:50)

                    at org.jboss.as.clustering.infinispan.invoker.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:34)

                    at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:87)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroupFactory$BeanGroupMutator.mutate(InfinispanBeanGroupFactory.java:109)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroup.close(InfinispanBeanGroup.java:120)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager.passivated(InfinispanBeanManager.java:249)

                    at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source) [:1.8.0_66]

                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_66]

                    at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_66]

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:207)

                    at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:229)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:192)

                    at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryPassivated(CacheNotifierImpl.java:374)

                    at org.infinispan.eviction.PassivationManagerImpl.passivate(PassivationManagerImpl.java:70)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:243)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:234)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.remove(BoundedConcurrentHashMap.java:1533)

            at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.removeEldestEntry(BoundedConcurrentHashMap.java:528)

                    at java.util.LinkedHashMap.afterNodeInsertion(LinkedHashMap.java:299) [rt.jar:1.8.0_66]

                    at java.util.HashMap.putVal(HashMap.java:663) [rt.jar:1.8.0_66]

                    at java.util.HashMap.put(HashMap.java:611) [rt.jar:1.8.0_66]

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.onEntryMiss(BoundedConcurrentHashMap.java:469)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.put(BoundedConcurrentHashMap.java:1427)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap.put(BoundedConcurrentHashMap.java:2025)

                    at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:170)

                    at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:163)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitCacheEntry(ClusteringDependentLogic.java:203)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$LocalLogic.commitEntry(ClusteringDependentLogic.java:277)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:317)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:542)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:303)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:98)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:78)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:83)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:114)

                    at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:101)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:154)

                    at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58)

                    at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:263)

                    at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:312)

                    at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:69)

                    at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBatcher$NewTransactionBatch.close(InfinispanBatcher.java:149)

                    at org.jboss.as.ejb3.cache.distributable.DistributableCache.create(DistributableCache.java:109) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.as.ejb3.component.stateful.StatefulComponentSessionIdGeneratingInterceptor.processInvocation(StatefulComponentSessionIdGeneratingInterceptor.java:57) [wildfly-ejb3-8.2.0.Final.

            jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

                    at org.jboss.as.ee.component.ViewService$DefaultViewInstanceFactory.createViewInstance(ViewService.java:292)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:177)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:173)

                    at org.jboss.as.ee.component.ViewManagedReferenceFactory.getReference(ViewManagedReferenceFactory.java:56)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:131)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:81)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:202)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)

                    at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:235)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:188)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacadeWildfly(AOMAServiceSessionBeanFactory.java:213) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacade(AOMAServiceSessionBeanFactory.java:183) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction$SimpleAction.loadSessionBean(SimpleDispatcherAction.java:555) [aoma.jar:]

                    at com.sonymusic.aoma.qc.notes.QCNotesVideoSaveAction.doAction(QCNotesVideoSaveAction.java:52) [aoma.jar:]

            at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction.executeAction(SimpleDispatcherAction.java:432) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet$DispatcherAction.execute(AOMADispatcherServlet.java:267) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet.service(AOMADispatcherServlet.java:80) [aoma.jar:]

                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

                    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:63) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final

            ]

                    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:247) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:76) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:166) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:197) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:759) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_66]

                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_66]

                    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_66]

             

             

            2016-02-08 14:52:55,265 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-22) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to ac

            quire lock after [120 seconds] on key [122c0375-f913-4acb-8502-a53c68f580e3] for requestor [GlobalTransaction:<null>:50751:local]! Lock held by [GlobalTransaction:<null>:48976:local]

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:291)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitReplaceCommand(PessimisticLockingInterceptor.java:179)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:255)

                    at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:201)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:139)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306)

                    at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:943)

                    at org.infinispan.CacheImpl.replace(CacheImpl.java:936)

                    at org.infinispan.DecoratedCache.replace(DecoratedCache.java:451)

                    at org.infinispan.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:246)

                    at org.jboss.as.clustering.infinispan.invoker.Mutator$MutateOperation.invoke(Mutator.java:50)

                    at org.jboss.as.clustering.infinispan.invoker.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:34)

                    at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:87)

            at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroupFactory$BeanGroupMutator.mutate(InfinispanBeanGroupFactory.java:109)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroup.close(InfinispanBeanGroup.java:120)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager.passivated(InfinispanBeanManager.java:249)

                    at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source) [:1.8.0_66]

                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_66]

                    at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_66]

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:207)

                    at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:229)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:192)

                    at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryPassivated(CacheNotifierImpl.java:374)

                    at org.infinispan.eviction.PassivationManagerImpl.passivate(PassivationManagerImpl.java:70)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:243)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:234)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.remove(BoundedConcurrentHashMap.java:1533)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.removeEldestEntry(BoundedConcurrentHashMap.java:528)

                    at java.util.LinkedHashMap.afterNodeInsertion(LinkedHashMap.java:299) [rt.jar:1.8.0_66]

                    at java.util.HashMap.putVal(HashMap.java:663) [rt.jar:1.8.0_66]

                    at java.util.HashMap.put(HashMap.java:611) [rt.jar:1.8.0_66]

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.onEntryMiss(BoundedConcurrentHashMap.java:469)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.put(BoundedConcurrentHashMap.java:1427)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap.put(BoundedConcurrentHashMap.java:2025)

                    at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:170)

                    at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:163)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitCacheEntry(ClusteringDependentLogic.java:203)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$LocalLogic.commitEntry(ClusteringDependentLogic.java:277)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:317)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:542)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:303)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:98)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:78)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:83)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:114)

                    at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:101)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:154)

                    at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58)

                    at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:263)

                    at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:312)

                    at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:69)

                    at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBatcher$NewTransactionBatch.close(InfinispanBatcher.java:149)

                    at org.jboss.as.ejb3.cache.distributable.DistributableCache.create(DistributableCache.java:109) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.as.ejb3.component.stateful.StatefulComponentSessionIdGeneratingInterceptor.processInvocation(StatefulComponentSessionIdGeneratingInterceptor.java:57) [wildfly-ejb3-8.2.0.Final.

            jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

                    at org.jboss.as.ee.component.ViewService$DefaultViewInstanceFactory.createViewInstance(ViewService.java:292)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:177)

            at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:173)

                    at org.jboss.as.ee.component.ViewManagedReferenceFactory.getReference(ViewManagedReferenceFactory.java:56)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:131)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:81)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:202)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)

                    at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:235)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:188)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacadeWildfly(AOMAServiceSessionBeanFactory.java:213) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacade(AOMAServiceSessionBeanFactory.java:183) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction$SimpleAction.loadSessionBean(SimpleDispatcherAction.java:555) [aoma.jar:]

                    at com.sonymusic.aoma.qc.notes.QCNotesVideoSaveAction.doAction(QCNotesVideoSaveAction.java:52) [aoma.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction.executeAction(SimpleDispatcherAction.java:432) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet$DispatcherAction.execute(AOMADispatcherServlet.java:267) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet.service(AOMADispatcherServlet.java:80) [aoma.jar:]

                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

                    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:63) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final

            ]

                    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:247) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:76) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:166) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:197) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:759) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_66]

                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_66]

                    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_66]

             

             

            2016-02-08 14:52:55,298 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-22) ISPN000136: Execution error: org.infinispan.commons.CacheListenerException: ISPN000280: Ca

            ught exception [org.infinispan.util.concurrent.TimeoutException] while invoking method [public void org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager.passivated(org.infinispan.notificatio

            ns.cachelistener.event.CacheEntryPassivatedEvent)] on listener instance: org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager@51823786

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:211)

                    at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:229)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:192)

                    at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryPassivated(CacheNotifierImpl.java:374)

                    at org.infinispan.eviction.PassivationManagerImpl.passivate(PassivationManagerImpl.java:70)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:243)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:234)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.remove(BoundedConcurrentHashMap.java:1533)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.removeEldestEntry(BoundedConcurrentHashMap.java:528)

                    at java.util.LinkedHashMap.afterNodeInsertion(LinkedHashMap.java:299) [rt.jar:1.8.0_66]

                    at java.util.HashMap.putVal(HashMap.java:663) [rt.jar:1.8.0_66]

                    at java.util.HashMap.put(HashMap.java:611) [rt.jar:1.8.0_66]

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.onEntryMiss(BoundedConcurrentHashMap.java:469)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.put(BoundedConcurrentHashMap.java:1427)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap.put(BoundedConcurrentHashMap.java:2025)

                    at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:170)

                    at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:163)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitCacheEntry(ClusteringDependentLogic.java:203)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$LocalLogic.commitEntry(ClusteringDependentLogic.java:277)

            at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:317)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:542)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:303)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:98)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:78)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:83)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:114)

                    at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:101)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:154)

                    at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58)

                    at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:263)

                    at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:312)

                    at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:69)

                    at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBatcher$NewTransactionBatch.close(InfinispanBatcher.java:149)

                    at org.jboss.as.ejb3.cache.distributable.DistributableCache.create(DistributableCache.java:109) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.as.ejb3.component.stateful.StatefulComponentSessionIdGeneratingInterceptor.processInvocation(StatefulComponentSessionIdGeneratingInterceptor.java:57) [wildfly-ejb3-8.2.0.Final.

            jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

                    at org.jboss.as.ee.component.ViewService$DefaultViewInstanceFactory.createViewInstance(ViewService.java:292)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:177)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:173)

                    at org.jboss.as.ee.component.ViewManagedReferenceFactory.getReference(ViewManagedReferenceFactory.java:56)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:131)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:81)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:202)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)

                    at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:235)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:188)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacadeWildfly(AOMAServiceSessionBeanFactory.java:213) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacade(AOMAServiceSessionBeanFactory.java:183) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction$SimpleAction.loadSessionBean(SimpleDispatcherAction.java:555) [aoma.jar:]

                    at com.sonymusic.aoma.qc.notes.QCNotesVideoSaveAction.doAction(QCNotesVideoSaveAction.java:52) [aoma.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction.executeAction(SimpleDispatcherAction.java:432) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet$DispatcherAction.execute(AOMADispatcherServlet.java:267) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet.service(AOMADispatcherServlet.java:80) [aoma.jar:]

                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

                    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

            at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:63) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final

            ]

                    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:247) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:76) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:166) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:197) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:759) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_66]

                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_66]

                    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_66]

            Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [120 seconds] on key [122c0375-f913-4acb-8502-a53c68f580e3] for requestor [GlobalTransaction:<null>:50751:lo

            cal]! Lock held by [GlobalTransaction:<null>:48976:local]

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:291)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitReplaceCommand(PessimisticLockingInterceptor.java:179)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:255)

                    at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:201)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:139)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1306)

                    at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:943)

                    at org.infinispan.CacheImpl.replace(CacheImpl.java:936)

                    at org.infinispan.DecoratedCache.replace(DecoratedCache.java:451)

                    at org.infinispan.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:246)

                    at org.jboss.as.clustering.infinispan.invoker.Mutator$MutateOperation.invoke(Mutator.java:50)

                    at org.jboss.as.clustering.infinispan.invoker.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:34)

                    at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:87)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroupFactory$BeanGroupMutator.mutate(InfinispanBeanGroupFactory.java:109)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroup.close(InfinispanBeanGroup.java:120)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager.passivated(InfinispanBeanManager.java:249)

                    at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source) [:1.8.0_66]

                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_66]

                    at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_66]

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:207)

                    ... 107 more

             

             

            2016-02-08 14:52:55,322 ERROR [org.infinispan.transaction.TransactionCoordinator] (default task-22) ISPN000097: Error while processing a prepare in a single-phase transaction: org.infinispan.common

            s.CacheListenerException: ISPN000280: Caught exception [org.infinispan.util.concurrent.TimeoutException] while invoking method [public void org.wildfly.clustering.ejb.infinispan.InfinispanBeanManag

            er.passivated(org.infinispan.notifications.cachelistener.event.CacheEntryPassivatedEvent)] on listener instance: org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager@51823786

            at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:211)

                    at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:22)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:229)

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation.invoke(AbstractListenerImpl.java:192)

                    at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryPassivated(CacheNotifierImpl.java:374)

                    at org.infinispan.eviction.PassivationManagerImpl.passivate(PassivationManagerImpl.java:70)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:243)

                    at org.infinispan.container.DefaultDataContainer$DefaultEvictionListener.onEntryChosenForEviction(DefaultDataContainer.java:234)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.remove(BoundedConcurrentHashMap.java:1533)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.removeEldestEntry(BoundedConcurrentHashMap.java:528)

                    at java.util.LinkedHashMap.afterNodeInsertion(LinkedHashMap.java:299) [rt.jar:1.8.0_66]

                    at java.util.HashMap.putVal(HashMap.java:663) [rt.jar:1.8.0_66]

                    at java.util.HashMap.put(HashMap.java:611) [rt.jar:1.8.0_66]

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$LRU.onEntryMiss(BoundedConcurrentHashMap.java:469)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap$Segment.put(BoundedConcurrentHashMap.java:1427)

                    at org.infinispan.util.concurrent.BoundedConcurrentHashMap.put(BoundedConcurrentHashMap.java:2025)

                    at org.infinispan.container.DefaultDataContainer.put(DefaultDataContainer.java:170)

                    at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:163)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitCacheEntry(ClusteringDependentLogic.java:203)

                    at org.infinispan.interceptors.locking.ClusteringDependentLogic$LocalLogic.commitEntry(ClusteringDependentLogic.java:277)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:317)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:542)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:303)

                    at org.infinispan.interceptors.EntryWrappingInterceptor.visitPrepareCommand(EntryWrappingInterceptor.java:98)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:78)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitPrepareCommand(PessimisticLockingInterceptor.java:83)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:36)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:114)

                    at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:101)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:96)

                    at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:124)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

                    at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:154)

                    at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58)

                    at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:263)

                    at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:312)

                    at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:69)

                    at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBatcher$NewTransactionBatch.close(InfinispanBatcher.java:149)

                    at org.jboss.as.ejb3.cache.distributable.DistributableCache.create(DistributableCache.java:109) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.as.ejb3.component.stateful.StatefulComponentSessionIdGeneratingInterceptor.processInvocation(StatefulComponentSessionIdGeneratingInterceptor.java:57) [wildfly-ejb3-8.2.0.Final.

            jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

                    at org.jboss.as.ee.component.ViewService$DefaultViewInstanceFactory.createViewInstance(ViewService.java:292)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:177)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:173)

                    at org.jboss.as.ee.component.ViewManagedReferenceFactory.getReference(ViewManagedReferenceFactory.java:56)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:131)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:81)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:202)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)

            at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:235)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:188)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacadeWildfly(AOMAServiceSessionBeanFactory.java:213) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacade(AOMAServiceSessionBeanFactory.java:183) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction$SimpleAction.loadSessionBean(SimpleDispatcherAction.java:555) [aoma.jar:]

                    at com.sonymusic.aoma.qc.notes.QCNotesVideoSaveAction.doAction(QCNotesVideoSaveAction.java:52) [aoma.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction.executeAction(SimpleDispatcherAction.java:432) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet$DispatcherAction.execute(AOMADispatcherServlet.java:267) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet.service(AOMADispatcherServlet.java:80) [aoma.jar:]

                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

                    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:63) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final

            ]

                    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:247) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:76) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:166) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:197) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:759) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_66]

                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_66]

                    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_66]

            Caused by: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [120 seconds] on key [122c0375-f913-4acb-8502-a53c68f580e3] for requestor [GlobalTransaction:<null>:50751:lo

            cal]! Lock held by [GlobalTransaction:<null>:48976:local]

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:198)

                    at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:171)

                    at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:169)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.lockAndRegisterBackupLock(PessimisticLockingInterceptor.java:291)

                    at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitReplaceCommand(PessimisticLockingInterceptor.java:179)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:112)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:255)

                    at org.infinispan.interceptors.TxInterceptor.visitReplaceCommand(TxInterceptor.java:201)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:148)

                    at org.infinispan.interceptors.CacheMgmtInterceptor.visitReplaceCommand(CacheMgmtInterceptor.java:139)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:110)

                    at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:73)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:98)

                    at org.infinispan.interceptors.BatchingInterceptor.handleDefault(BatchingInterceptor.java:66)

                    at org.infinispan.commands.AbstractVisitor.visitReplaceCommand(AbstractVisitor.java:42)

                    at org.infinispan.commands.write.ReplaceCommand.acceptVisitor(ReplaceCommand.java:55)

                    at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:333)

            at org.infinispan.CacheImpl.replaceInternal(CacheImpl.java:943)

                    at org.infinispan.CacheImpl.replace(CacheImpl.java:936)

                    at org.infinispan.DecoratedCache.replace(DecoratedCache.java:451)

                    at org.infinispan.AbstractDelegatingCache.replace(AbstractDelegatingCache.java:246)

                    at org.jboss.as.clustering.infinispan.invoker.Mutator$MutateOperation.invoke(Mutator.java:50)

                    at org.jboss.as.clustering.infinispan.invoker.SimpleCacheInvoker.invoke(SimpleCacheInvoker.java:34)

                    at org.jboss.as.clustering.infinispan.invoker.RetryingCacheInvoker.invoke(RetryingCacheInvoker.java:87)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroupFactory$BeanGroupMutator.mutate(InfinispanBeanGroupFactory.java:109)

                    at org.wildfly.clustering.ejb.infinispan.group.InfinispanBeanGroup.close(InfinispanBeanGroup.java:120)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBeanManager.passivated(InfinispanBeanManager.java:249)

                    at sun.reflect.GeneratedMethodAccessor104.invoke(Unknown Source) [:1.8.0_66]

                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_66]

                    at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_66]

                    at org.infinispan.notifications.AbstractListenerImpl$ListenerInvocation$1.run(AbstractListenerImpl.java:207)

                    ... 107 more

             

             

            2016-02-08 14:52:55,345 ERROR [org.infinispan.transaction.tm.DummyTransaction] (default task-22) ISPN000111: afterCompletion() failed for SynchronizationAdapter{localTransaction=LocalTransaction{re

            moteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, topologyId=-1, isFromStateTransfer=false} org.infinispan.transaction.synchronization.SyncLocalTransaction@bf50

            } org.infinispan.transaction.synchronization.SynchronizationAdapter@bf6f: org.infinispan.commons.CacheException: Could not commit.

                    at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:60)

                    at org.infinispan.transaction.tm.DummyTransaction.notifyAfterCompletion(DummyTransaction.java:263)

                    at org.infinispan.transaction.tm.DummyTransaction.runCommitTx(DummyTransaction.java:312)

                    at org.infinispan.transaction.tm.DummyTransaction.commit(DummyTransaction.java:69)

                    at org.infinispan.transaction.tm.DummyBaseTransactionManager.commit(DummyBaseTransactionManager.java:80)

                    at org.wildfly.clustering.ejb.infinispan.InfinispanBatcher$NewTransactionBatch.close(InfinispanBatcher.java:149)

                    at org.jboss.as.ejb3.cache.distributable.DistributableCache.create(DistributableCache.java:109) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.as.ejb3.component.stateful.StatefulComponentSessionIdGeneratingInterceptor.processInvocation(StatefulComponentSessionIdGeneratingInterceptor.java:57) [wildfly-ejb3-8.2.0.Final.

            jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

                    at org.jboss.as.ee.component.ViewService$DefaultViewInstanceFactory.createViewInstance(ViewService.java:292)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:177)

                    at org.jboss.as.ee.component.ViewService$View.createInstance(ViewService.java:173)

                    at org.jboss.as.ee.component.ViewManagedReferenceFactory.getReference(ViewManagedReferenceFactory.java:56)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:131)

                    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:81)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:202)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:179)

                    at org.jboss.as.naming.InitialContext$DefaultInitialContext.lookup(InitialContext.java:235)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:188)

                    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:184)

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at javax.naming.InitialContext.lookup(InitialContext.java:417) [rt.jar:1.8.0_66]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacadeWildfly(AOMAServiceSessionBeanFactory.java:213) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.service.AOMAServiceSessionBeanFactory.getLocalFacade(AOMAServiceSessionBeanFactory.java:183) [aoma.service.ejb-interfaces.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction$SimpleAction.loadSessionBean(SimpleDispatcherAction.java:555) [aoma.jar:]

                    at com.sonymusic.aoma.qc.notes.QCNotesVideoSaveAction.doAction(QCNotesVideoSaveAction.java:52) [aoma.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction.executeAction(SimpleDispatcherAction.java:432) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet$DispatcherAction.execute(AOMADispatcherServlet.java:267) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet.service(AOMADispatcherServlet.java:80) [aoma.jar:]

                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

                    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:63) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final

            ]

                    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.security.handlers.SecurityInitialHandler.handleRequest(SecurityInitialHandler.java:76) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:261) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

            at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:247) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:76) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:166) [undertow-servlet-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.Connectors.executeRootHandler(Connectors.java:197) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:759) [undertow-core-1.1.0.Final.jar:1.1.0.Final]

                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_66]

                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_66]

                    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_66]

            Caused by: javax.transaction.xa.XAException

                    at org.infinispan.transaction.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:204)

                    at org.infinispan.transaction.TransactionCoordinator.commit(TransactionCoordinator.java:156)

                    at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:58)

                    ... 56 more

             

             

            2016-02-08 14:52:55,354 ERROR [org.jboss.as.ejb3.invocation] (default task-22) JBAS014134: EJB Invocation failed on component VideoMetadataAndQCNotesEditor for method public abstract void com.sonym

            usic.aoma.service.ejb.entityfacade.local.VideoMetadataAndQCNotesEditorLocal.startEditing(java.lang.Long,java.lang.Long): javax.ejb.NoSuchEJBException: JBAS014300: Could not find EJB with id {[-44,

            -16, -114, 68, 96, -11, 68, -52, -101, -98, -58, 24, -111, 89, -121, 92]}

                    at org.jboss.as.ejb3.component.stateful.StatefulComponentInstanceInterceptor.processInvocation(StatefulComponentInstanceInterceptor.java:62) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:260) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.as.ejb3.tx.CMTTxInterceptor.notSupported(CMTTxInterceptor.java:324) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:237) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:43) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:95) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64)

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:326)

                    at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:448)

                    at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61)

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:326)

                    at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80)

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

                    at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185)

                    at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182)

                    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)

                    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)

                    at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73)

                    at com.sonymusic.aoma.service.ejb.entityfacade.local.VideoMetadataAndQCNotesEditorLocal$$$view182.startEditing(Unknown Source) [aoma.service.local.jar:]

                    at com.sonymusic.aoma.qc.notes.QCNotesVideoSaveAction.doAction(QCNotesVideoSaveAction.java:53) [aoma.jar:]

                    at com.sonymusic.aoma.dispatcher.actions.SimpleDispatcherAction.executeAction(SimpleDispatcherAction.java:432) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet$DispatcherAction.execute(AOMADispatcherServlet.java:267) [aoma.jar:]

                    at com.sonymusic.aoma.AOMADispatcherServlet.service(AOMADispatcherServlet.java:80) [aoma.jar:]

                    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) [jboss-servlet-api_3.1_spec-1.0.0.Final.jar:1.0.0.Final]

                    at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85)

                    at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:61)

                    at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)

                    at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)

                    at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131)

                    at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:56)

                    at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)

                    at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:45)

                    at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:63)

                    at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:58)

                    at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:70)

            • 3. Re: Timeout Exception when using EJB Infinispan cache
              arnab_ghosh

              Is it possible that this is happening because of the cache being non-transactional ?

              • 4. Re: Timeout Exception when using EJB Infinispan cache
                pferraro

                Which version of WildFly are you using?

                • 5. Re: Timeout Exception when using EJB Infinispan cache
                  arnab_ghosh

                  Paul,

                   

                  We are using Wildfly 8.2.0.Final. Please let me know if you need more information.

                   

                  Regards

                  Arnab

                  • 6. Re: Timeout Exception when using EJB Infinispan cache
                    pferraro

                    Don't attempt to use infinispan-based expiration/eviction, for a number of reasons:

                    1. The distributed SFSB cache implementation already handles expiration and passivation.  Expiration behavior defined per bean via the @javax.ejb.StatefulTimeout annotation.  Passivation behavior is defined by the ejb3 subsystem configuration.
                    2. The distributed SFSB cache implementation maps a SFSB to multiple cache entries - thus a set of entries must expire/passivate/activate as a unit.  Infinispan's expiration and eviction implementations operate on individual cache entries.
                    • 7. Re: Timeout Exception when using EJB Infinispan cache
                      arnab_ghosh

                      Paul,

                       

                      Thanks for your reply. I have below doubts

                      1) We don't have a requirement of a clustered cache set up or need of SFSB passivation. I moved to distributable cache from simple cache, because we are observing OOM issues, and on analysis of the heap, it seems to be full of SFSB instances.  Is there any other type of cache  which I should use?

                      2) Currently we don't use @javax.ejb.StatefulTimeout . Do you think we re better off using simple cache and using stateful timeout on SFSB  or should I continue using distributable cache but without infinispan eviction/expiration along with stateful timeout on SFSB:

                           <cache-container name="ejb" default-cache="passivation" module="org.wildfly.clustering.ejb.infinispan" aliases="sfsb" statistics-enabled="true">  

                      <local-cache name="passivation" batching="true" statistics-enabled="true" >  

                        <file-store passivation="true" purge="true" />  

                        </local-cache>

                      • 8. Re: Timeout Exception when using EJB Infinispan cache
                        pferraro

                        arnab_ghosh

                        1) If you have no need for your SFSB state to be HA then why not use @Stateful(passivation-capable=false)?  This will use the SFSB cache defined by the passivation-disabled-cache-ref configuration (as defined in the ejb3 subsystem)  That way WF will never try to replicate or passivate your SFSB state.

                        2) If you do not need passivation/replication, then you should definitely disable it explicitly (see #1).  The simple cache implementation will always be more efficient than the distributable one.  In general, SFSBs will always remain in memory until they are removed - either by explicit invocation of a @Remove method, or after a specified @StatefulTimeout, or by making them contextual with an associated scope (e.g. @RequestScoped, @SessionScoped, @ApplicationScoped, etc.).

                        • 9. Re: Timeout Exception when using EJB Infinispan cache
                          arnab_ghosh

                          pferraro Thanks for the explanation. Now I understand this much better. I will implement this right away. I will let you know how it goes.

                           

                          Regards

                          Arnab

                          • 10. Re: Timeout Exception when using EJB Infinispan cache
                            arnab_ghosh

                            pferraro

                             

                            Now I have set the @statefultimeout for all SessionBeans and reverted to using simple cache.

                             

                            I am monitoring the a single SessionBean using the below CLI

                            /deployment=aoma.ear/subdeployment=aoma.service.local.jar/subsystem=ejb3/stateful-session-bean=UserFacade:read-resource(recursive=true, include-runtime=true)

                             

                            is there a better way to monitor the cache ?

                             

                            Regards

                            Arnab

                            • 11. Re: Timeout Exception when using EJB Infinispan cache
                              arnab_ghosh

                              pferraro

                               

                              I ran into the heap problem again after almost 6 days. The report still shows a bunch of simple cache object. Attaching a report from Eclipse memory analysis tool.

                              I thought setting @Statefultimeout should evicts of all SFSB instances from the simplecache. Please let me know if you have any suggestions.

                               

                              Regards

                              Arnab

                              • 12. Re: Timeout Exception when using EJB Infinispan cache
                                pferraro

                                From @StatefulTimeout javadoc:

                                "Specifies the amount of time a stateful session bean [instance] can be idle (not receive any client invocations) before it is eligible for removal by the container."

                                 

                                This doesn't remove all SFSB instances, only those that have been idle for the period of time specified by the annotation (or descriptor).

                                 

                                If you find that you are creating more SFSBs than can fit in memory, you should consider one or more of the following:

                                • Ensure that you are not maintaining any strong references to your SFSB instance from some longer living object preventing the memory allocated for a SFSB instance from being freed after removal
                                • Allocating more system memory to your JVM heap
                                • Proactively @Remove your SFSB instances when they are no longer needed
                                • Consider a shorter @StatefulTimeout
                                • Use passivation as a strategy to free up memory of least recently used SFSB instances
                                • Distributing SFSB load across multiple nodes
                                • 13. Re: Timeout Exception when using EJB Infinispan cache
                                  arnab_ghosh

                                  Paul, Thanks for your reply. Please see my comments inline.

                                   

                                  • Ensure that you are not maintaining any strong references to your SFSB instance from some longer living object preventing the memory allocated for a SFSB instance from being freed after removal - I will look into this and see if something keeping a strong reference.
                                  • Proactively @Remove your SFSB instances when they are no longer needed - The code does invoke @Remove method once the UI request is completed. The remove method of every SFSB invokes remove method on all the child SessionBeans that it injects. This clean up code may not completely accurate since this is a legacy app, so I was expecting after 15 mins of IDLE time out the old SFSB's will be removed.
                                  • Use passivation as a strategy to free up memory of least recently used SFSB instances -  Can you please elaborate how I can set this up ? Please point me to any documentation available.

                                   

                                  Another thing worth mentioning here is, this application was getting deployed in JBoss EAP 5.1.2 previously and we migrated it now to Wildfly 8.2.x. There was no memory issues before the migration. Please let me know if you need any configuration files from our old server for better understanding.

                                   

                                  Regards

                                  Arnab

                                  • 14. Re: Timeout Exception when using EJB Infinispan cache
                                    arnab_ghosh

                                    pferraro

                                     

                                    In our old server configuration, in ejb3-interceptors-aop.xml, I am seeing the below:

                                     

                                          <!-- NON Clustered cache configuration -->

                                          <annotation expr="class(*) AND !class(@org.jboss.ejb3.annotation.Cache) AND !class(@org.jboss.ejb3.annotation.Clustered)">

                                             @org.jboss.ejb3.annotation.Cache ("SimpleStatefulCache")

                                          </annotation>

                                          <annotation expr="class(*) AND !class(@org.jboss.ejb3.annotation.PersistenceManager) AND !class(@org.jboss.ejb3.annotation.Clustered)">

                                             @org.jboss.ejb3.annotation.PersistenceManager ("StatefulSessionFilePersistenceManager")

                                          </annotation>

                                          <annotation expr="class(*) AND !class(@org.jboss.ejb3.annotation.CacheConfig) AND !class(@org.jboss.ejb3.annotation.Clustered)">

                                             @org.jboss.ejb3.annotation.CacheConfig (maxSize=150000, idleTimeoutSeconds=3600, removalTimeoutSeconds=900)

                                          </annotation>

                                     

                                    Is there a way I can configure similar parameters ? Please let me know.

                                    1 2 Previous Next