Workspace Cache and Transactions

[bwallis42]

If I configure a workspace cache with NON_XA transactions I get an error at startup (stacktrace below).

I can make a minor change to the out of the box configuration of the sample repository and get the error when I access the repository via the rest interface.

I'm adding a workspace cache container and cache like so:

{code}            <cache-container name="modeshapeWorkspaceCache">
                <local-cache name="sample/default">
                    <transaction mode="NON_XA"/>
                    <eviction strategy="LRU" max-entries="10000"/>
                    <expiration max-idle="5000" lifespan="10000" interval="1000"/>
                </local-cache>
            </cache-container>
{code}

and modifying the sample repository definition like so:

{code}            <repository name="sample">
                    <workspaces cache-container="modeshapeWorkspaceCache">
                        <workspace name="default"/>
                    </workspaces>
            </repository>
{code}

If I change the workspace cache transaction mode to "NONE" then the error goes away.

This problem started occuring in my own project and I simplified it to this. I've just updated my 3.2-SNAPSHOT this morning and this seems to be what has introduced the error. My last update was mid last week. Running JDK 1.7.0_17 on Mac OSX 10.8.

{code}
14:55:27,309 INFO  [org.jboss.as.clustering.infinispan] (modeshape-indexer-8-thread-1) JBAS010281: Started sample/default cache from modeshapeWorkspaceCache container
14:55:27,324 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (modeshape-indexer-8-thread-1) ISPN000136: Execution error: java.lang.IllegalStateException: Transaction TransactionImple < ac, BasicAction: 0:ffff0a402145:430014fa:516b8817:1b status: ActionStatus.COMMITTED > is not in a valid state to be invoking cache operations on.
          at org.infinispan.interceptors.TxInterceptor.enlist(TxInterceptor.java:268) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:245) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.TxInterceptor.visitRemoveCommand(TxInterceptor.java:201) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:72) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.CacheMgmtInterceptor.visitRemoveCommand(CacheMgmtInterceptor.java:137) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:72) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:67) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:72) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1162) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.CacheImpl.removeInternal(CacheImpl.java:308) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.CacheImpl.remove(CacheImpl.java:302) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.CacheImpl.remove(CacheImpl.java:296) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:313) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.modeshape.jcr.cache.document.WorkspaceCache.changed(WorkspaceCache.java:299) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.cache.document.TransactionalWorkspaceCache.changed(TransactionalWorkspaceCache.java:73) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.txn.SynchronizedTransactions$WorkspaceUpdates.apply(SynchronizedTransactions.java:219) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.txn.SynchronizedTransactions$SynchronizedTransaction.afterCommit(SynchronizedTransactions.java:171) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.txn.SynchronizedTransactions$SynchronizedTransaction$1.afterCompletion(SynchronizedTransactions.java:127) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
          at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:402)
          at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103)
          at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
          at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
          at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
          at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
          at org.modeshape.jcr.txn.Transactions$SimpleTransaction.commit(Transactions.java:283) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.cache.RepositoryCache.runInTransaction(RepositoryCache.java:315) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.cache.RepositoryCache.workspace(RepositoryCache.java:575) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.cache.RepositoryCache.getWorkspaceCache(RepositoryCache.java:761) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.RepositoryQueryManager.reindexContent(RepositoryQueryManager.java:240) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.RepositoryQueryManager.access$000(RepositoryQueryManager.java:72) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.RepositoryQueryManager$1.call(RepositoryQueryManager.java:208) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.RepositoryQueryManager$1.call(RepositoryQueryManager.java:204) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_17]
          at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_17]
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_17]
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_17]
          at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_17]
14:55:27,332 WARN  [com.arjuna.ats.jta] (modeshape-indexer-8-thread-1) ARJUNA016029: SynchronizationImple.afterCompletion - failed for org.modeshape.jcr.txn.SynchronizedTransactions$SynchronizedTransaction$1@6892fef5 with exception: java.lang.IllegalStateException: Transaction TransactionImple < ac, BasicAction: 0:ffff0a402145:430014fa:516b8817:1b status: ActionStatus.COMMITTED > is not in a valid state to be invoking cache operations on.
          at org.infinispan.interceptors.TxInterceptor.enlist(TxInterceptor.java:268) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:245) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.TxInterceptor.visitRemoveCommand(TxInterceptor.java:201) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:72) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.CacheMgmtInterceptor.visitRemoveCommand(CacheMgmtInterceptor.java:137) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:72) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:128) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:92) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.commands.AbstractVisitor.visitRemoveCommand(AbstractVisitor.java:67) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:72) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:343) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1162) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.CacheImpl.removeInternal(CacheImpl.java:308) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.CacheImpl.remove(CacheImpl.java:302) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.CacheImpl.remove(CacheImpl.java:296) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.infinispan.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:313) [infinispan-core-5.2.1.Final.jar:5.2.1.Final]
          at org.modeshape.jcr.cache.document.WorkspaceCache.changed(WorkspaceCache.java:299) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.cache.document.TransactionalWorkspaceCache.changed(TransactionalWorkspaceCache.java:73) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.txn.SynchronizedTransactions$WorkspaceUpdates.apply(SynchronizedTransactions.java:219) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.txn.SynchronizedTransactions$SynchronizedTransaction.afterCommit(SynchronizedTransactions.java:171) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.txn.SynchronizedTransactions$SynchronizedTransaction$1.afterCompletion(SynchronizedTransactions.java:127) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96)
          at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:402)
          at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103)
          at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)
          at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165)
          at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126)
          at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
          at org.modeshape.jcr.txn.Transactions$SimpleTransaction.commit(Transactions.java:283) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.cache.RepositoryCache.runInTransaction(RepositoryCache.java:315) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.cache.RepositoryCache.workspace(RepositoryCache.java:575) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.cache.RepositoryCache.getWorkspaceCache(RepositoryCache.java:761) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.RepositoryQueryManager.reindexContent(RepositoryQueryManager.java:240) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.RepositoryQueryManager.access$000(RepositoryQueryManager.java:72) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.RepositoryQueryManager$1.call(RepositoryQueryManager.java:208) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at org.modeshape.jcr.RepositoryQueryManager$1.call(RepositoryQueryManager.java:204) [modeshape-jcr-3.2-SNAPSHOT.jar:3.2-SNAPSHOT]
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_17]
          at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_17]
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_17]
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_17]
          at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_17]
{code}
14:55:27,337 WARN  [com.arjuna.ats.arjuna] (modeshape-indexer-8-thread-1) ARJUNA012127: TwoPhaseCoordinator.afterCompletion - returned failure for SynchronizationImple< 0:ffff0a402145:430014fa:516b8817:1f, org.modeshape.jcr.txn.SynchronizedTransactions$SynchronizedTransaction$1@6892fef5 >

[rhauch]

The workspace cache should NOT use transactions. It's used internally to cache already persisted content, and never exposed to client applications. The code that uses it is threadsafe.

The rest of your configuration looks good. Of course you may want to tweak the expiration and eviction attributes. You're using LRU, but the Infinispan documentation suggests that LIRS doesn't have some of the drawbacks of LRU. Give both a try to see which one works best for you.

[rhauch]

Horia opened up MODE-1899 to check that transactions are not enabled, and if they are to produce an error (with meaningful message) upon startup.

[bwallis42]

Thanks for that.

I've not yet started to play with the settings, it is just a placeholder in my config for now to remind me that I need to investigate the effect on performance with our workload and content.

I have successfully run some early performance tests that we also ran back in 2.8 days and I have to say it is looking good. With 2.8 the performance tended to decrease as the repository grew but with 3.2 it seems to stay pretty flat up to the 160000 nodes I've tried so far (with the exception of the JOIN expressions in some queries in our test). Will hopefully get a dedicated performance machine setup soon rather than trying to get sensible results on my overloaded laptop :-)

[bwallis42]

Randall Hauch wrote:

Horia opened up MODE-1899 to check that transactions are not enabled, and if they are to produce an error (with meaningful message) upon startup.

Thanks for that, It is always a timesaver when a system rejects an invalid configuration early on with a sensible error message pointing to the problem.