8 Replies Latest reply on Jan 31, 2006 12:34 PM by Manik Surtani

    Transactions in a replicated cache

    Apprentice

      I'm trying to cache several nodes within a transaction in a replicated cache environment. I'm using the latest JBossCache HEAD code along with a fairly recent build of JBossAS HEAD. The transaction is being executed from a servlet running on one of the cluster nodes.

      If I use a non-replicated environment (e.g., INVALIDATION_SYNC), the transaction executes properly and commits or rolls back as expected.

      If I use a replicated environment (e.g., REPL_SYNC), the transaction executes properly as long as I configure the cache to use the DummyTransactionManager and I use this manager to execute the transaction from the servlet.

      If I get a transaction manager from JNDI (either java:TransactionManager or UserTransaction) and I configure my cache to use JBossTransactionManagerLookup, the transaction always fails during the prepare phase on the remote node. As noted, the same code works properly if the cache isn't replicated. I've attached the relevant stack trace.

      I've looked for a JBossCache test that executes similarly (i.e., transaction in a replicated environment) but I've only been able to find one that executes in standalone mode using the dummy manager.

      Any ideas on whether I'm missing something here?

      ERROR [RpcDispatcher] failed invoking method
      org.jboss.util.NestedRuntimeException: Already associated with a tx; - nested throwable: (java.lang.IllegalStateException: Already associated with a tx)
      at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:4247)
      at org.jboss.cache.TreeCache._replicate(TreeCache.java:3894)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:324)
      at org.jgroups.blocks.MethodCall.invoke(MethodCall.java:286)
      at org.jgroups.blocks.RpcDispatcher.handle(RpcDispatcher.java:236)
      at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:618)
      at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:515)
      at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:326)
      at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUp(MessageDispatcher.java:734)
      at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.access$300(MessageDispatcher.java:566)
      at org.jgroups.blocks.MessageDispatcher$1.run(MessageDispatcher.java:703)
      at java.lang.Thread.run(Thread.java:534)
      Caused by: java.lang.IllegalStateException: Already associated with a tx
      at org.jboss.tm.TxManager.resume(TxManager.java:660)
      at org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:263)
      at org.jboss.cache.interceptors.TxInterceptor.invoke(TxInterceptor.java:124)
      at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:67)
      at org.jboss.cache.interceptors.CacheMgmtInterceptor.invoke(CacheMgmtInterceptor.java:128)
      at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:4241)




        • 1. Re: Transactions in a replicated cache
          Manik Surtani Master

          From Brian's response email:

          "brian.stansberry@jboss.com" wrote:
          Copying Manik.

          I think this is because JBoss's TxManager.resume(Transaction tx) checks
          if there is an existing uncompleted tx bound to the thread, and if there
          is throws an exception. It doesn't check if the tx passed as a method
          arg is the same as the one already bound to the thread.

          Here's the relevant code from TxInterceptor.handleRemotePrepare()

          if (ltx == null)
          {
          ltx = createLocalTxForGlobalTx(gtx); // creates new LTX
          and associates it with a GTX
          if (log.isDebugEnabled())
          {
          log.debug("(" + cache.getLocalAddress() + "):
          started new local TX as result of remote PREPARE: local TX=" + ltx + ",
          global TX=" + gtx);
          }
          }
          else
          {
          //this should be valid
          if (!isValid(ltx)) throw new CacheException("Transaction
          " + ltx + " not in correct state to be prepared");
          }

          //associate this thread with this ltx
          txManager.resume(ltx);
          if (log.isTraceEnabled()) {log.trace("Resuming existing
          transaction " + ltx + ", global TX=" + gtx);}


          I think if the last couple lines were inside the preceding else block
          the problem would go away.

          Good job on this one, Jerry. Clearly we need some integration tests that
          exercise JBossCache with JBoss' transaction manager. Not easy in
          standalone mode, as JBoss AS is required. The HttpSession replication
          tests used to do that, but now they all use BatchModeTransactionManager,
          which is a subclass of DummyTransactionManager.


          Brian


          • 2. Re: Transactions in a replicated cache
            Manik Surtani Master

            Thanks Brian and Jerry, I'll look into and test a fix for this.

            This is what releasing a DR is supposed to check. I expect QA will spot this one as well, when running through the latest AS test suite. :)

            • 3. Re: Transactions in a replicated cache
              Manik Surtani Master

              Ok, I've fixed this in HEAD. Expecting to hear from QA with this problem in 1.3.0.DR1 any minute now! :-) We do need some integration tests, the question is where should such tests reside. I dare say they should be in the AS test suite. And such tests probably already do exist in AS.

              I also spotted something else that has changed with this version of JBossCache: with 1.2.4, I could write a test that created 2 replicated cache instances in the same JBossAS instance, and I could run these tx tests. With 1.3.0, I had to fire up 2 separate AS instances, and run one treecache instance in each. The problem was to do with JMX bindings and the new CacheMgmtInterceptors - the JMX server threw up when trying to bind 2 cache instances which were supposed to be in the same cluster since the cluster name was used as the service name. I don't think this is a "real world" problem at all, although I think it may break some AS tests.

              • 4. Re: Transactions in a replicated cache
                Apprentice

                The fix seems to be fine. At least my own sample code having transactions in a replicated cache now works. :)

                re: the multiple instance issue - I recently modified the interceptor mbean registration code so that the mbean will only be registered if not already registered. So the server should no longer regurgitate but it's still not going to do what you need. It seems like you would need to define two separate instances of the cache having different service names. You can also disable the interceptor mbeans by setting UseInterceptorMbeans to false in the cache's configuration.

                • 5. Re: Transactions in a replicated cache
                  Apprentice

                  I've just tried using two clustered cache instances on a single server and it seemed to run fine. I created the second cache configuration by copying the first and then modifying the service name. The instances were registered properly and replication worked as expected.

                  • 6. Re: Transactions in a replicated cache
                    Brian Stansberry Master

                     

                    "manik.surtani@jboss.com" wrote:
                    Ok, I've fixed this in HEAD. Expecting to hear from QA with this problem in 1.3.0.DR1 any minute now! :-) We do need some integration tests, the question is where should such tests reside. I dare say they should be in the AS test suite. And such tests probably already do exist in AS.


                    I don't believe so :( The AS tests I'm aware of relate to HttpSession replication and clustered SSO, and those now use a cache that uses a DummyTransactionManager subclass as its TM. There may be some EJB3 related tests that use JBoss' TM; I'm not sure.

                    I'll open a JIRA issue for this. It will be an AS issue; I guess the corresponding JBossCache issue is that a release has to pass the unit tests for whatever AS version(s) it's supposed to integrate with. It's great you're already pushing this with 1.3.0, Manik.

                    • 8. Re: Transactions in a replicated cache
                      Manik Surtani Master

                       

                      "bstansberry@jboss.com" wrote:
                      It's great you're already pushing this with 1.3.0, Manik.


                      Integration woes, great? I don't think so! :-)