4 Replies Latest reply on May 15, 2006 8:27 AM by marklittle

    Recovery of a single XAResource

    paul.robinson

      Hello,

      I have a single XAResource that I have implemented myself. Each method simply logs the fact that it has been called. I have also added readObject and writeObject methods to log when serialization and de-serialization occurs (these call the defaultReadObject and defaultWriteObject methods).

      My XAResource kills JBoss when commit is called. Thus commit never returns and thus, I think, the recovery manager should re-call comit when JBoss comes back up. However, it doesn't!

      This is the output of the one phase commit on my XAResource:

      15:34:20,985 INFO [STDOUT] C: start
      15:34:20,985 INFO [STDOUT] C: end
      15:34:20,985 INFO [STDOUT] C: commit
      15:34:20,985 INFO [STDOUT] C: CRASH!

      So, the start method is called on the XAResource, followed by the end method and then the commit method. Prepare is not called as one-phase commit is in operation. Interestingly, the object is not serialized. Thus not added to the intentions list. So, when JBoss comes back to life, there is no serialized XAResource to commit.

      Is this valid behaviour?

      When I have 3 of my XAResources, recovery works. Here two-phase commit is in operation and I can see each of my resources being serialized to the intentions list. Thus I'm confident that I am doing the right thing in my XAResource.

      Thanks,

      Paul.

        • 1. Re: Recovery of a single XAResource
          marklittle

           

          "n9086822" wrote:
          Hello,

          I have a single XAResource that I have implemented myself. Each method simply logs the fact that it has been called. I have also added readObject and writeObject methods to log when serialization and de-serialization occurs (these call the defaultReadObject and defaultWriteObject methods).

          My XAResource kills JBoss when commit is called. Thus commit never returns and thus, I think, the recovery manager should re-call comit when JBoss comes back up. However, it doesn't!

          This is the output of the one phase commit on my XAResource:

          15:34:20,985 INFO [STDOUT] C: start
          15:34:20,985 INFO [STDOUT] C: end
          15:34:20,985 INFO [STDOUT] C: commit
          15:34:20,985 INFO [STDOUT] C: CRASH!

          So, the start method is called on the XAResource, followed by the end method and then the commit method. Prepare is not called as one-phase commit is in operation. Interestingly, the object is not serialized. Thus not added to the intentions list. So, when JBoss comes back to life, there is no serialized XAResource to commit.

          Is this valid behaviour?

          When I have 3 of my XAResources, recovery works. Here two-phase commit is in operation and I can see each of my resources being serialized to the intentions list. Thus I'm confident that I am doing the right thing in my XAResource.

          Thanks,

          Paul.


          What you are seeing is the one-phase commit optimization at work. With only a single resource, there is no need to run the 2PC protocol, since consensus is implicity: whatever the resource says, is the outcome of the transaction. If you have implemented the XAResource correctly, then you will never see commit being called. What you'll see being invoked is commitOnePhase (unless you disable the one-phase commit optimization).

          You should read the JBossTS manuals to familiarise yourself with the capabilities of JBossTS itself and distributed transactions in general.

          • 2. Re: Recovery of a single XAResource
            marklittle

            In case my previous post wasn't clear enough (and you will find all of this within the manuals): there is no need to create any log information for a single resource, so no entry will appear in the object store. Your XAResource will not be asked to serialize itself. The coordinator will simply invoke commitOnePhase and return the response to the application (i.e., transaction committed or rolled back).

            • 3. Re: Recovery of a single XAResource
              paul.robinson

              Thanks for the response,

              I think you misunderstood my question. I am familiar with the one-phase commit optimization; that is what I was testing at the time.

              I was expectng the XAResource to be serialized to the object store before commitOnePhase is called, thus allowing it to be recovered in case a crash occurs before the coordinator receives the outcome of the commitOnePhase execution.

              So, if the system crashes before the return from the commitOnePhase method, what will happen? As the commitOnePhase method will not be re-called when the system recovers, the coordinator will not know whether the commitOnePhase method completed.

              I understand that the system will be left in a consistant state, as there is only one resource. However, I don't think the client or coordinator will know what the outcome is.

              Paul.

              • 4. Re: Recovery of a single XAResource
                marklittle

                 

                "n9086822" wrote:
                Thanks for the response,

                I think you misunderstood my question. I am familiar with the one-phase commit optimization; that is what I was testing at the time.


                In two phase commit, we use multiple phases (2) to ensure that we obtain consensus between the various participants. At the end of the first phase, both participants and coordinator have written some durable representation of their state, which is sufficient to replay in the event of failure.

                In the 1PC optimization, we implicitly have consensus, so there is no need to run multiple phases: the outcome of the transaction is no longer the domain of the coordinator, but solely on the single registered participant. As a result, there is no need for the coordinator to write a log entry because it will be of little use in the event of failure and recovery - the participant won't have written a corresponding entry. This means that (ignoring heuristics, which can happen anyway), even if the coordinator remembers the participant, upon recovery if it tries to issue commit again, it will find that the participant has gone away and can't know whether it committed or rolled back - there will be no state at the participant side to indicate one way or another (typically).

                This is an optimization for the most common case: success. Failures don't happen that often and we don't want to optimize for that. In this small window of vulernability, we're talking about a situation where the coordinator AND participant crash at commit_one_phase and no response goes back to the application. The application can subsequently figure out what happened (presumably), but it will know that there is no state corruption, irrespective of the outcome.


                I was expectng the XAResource to be serialized to the object store before commitOnePhase is called, thus allowing it to be recovered in case a crash occurs before the coordinator receives the outcome of the commitOnePhase execution.


                Hopefully I've explained why that's not the case. There's a really good book (http://www.amazon.com/gp/product/013035290X/ref=sr_11_1/103-3905243-8795858?%5Fencoding=UTF8) that explains it more ;-)


                So, if the system crashes before the return from the commitOnePhase method, what will happen? As the commitOnePhase method will not be re-called when the system recovers, the coordinator will not know whether the commitOnePhase method completed.


                If the coordinator does not crash, then it is in doubt. Depending on the implementation, it'll either indicate a heuristic (hazard) or that the transaction rolled back (depends what the failure was, but this is tricky to ensure).


                I understand that the system will be left in a consistant state, as there is only one resource. However, I don't think the client or coordinator will know what the outcome is.

                Paul.


                Yes, that's correct. But it's an edge case and worth it for the optimization.

                If you don't want the optimization, JBossTS allows you to disable the one-phase commit for pretty much these precise reasons.