[tomjenkinson]

Hi David,

No, its not quite that, that scenario you describe does exist, but thats not the scenario termed an orphan in this case.

An orphan is where a remote resource has a record of the transaction but the remote transaction doesn't.

Sequence:

1. Enlist resources at TM1

2. Propagate to TM2

3. Enlist resources at TM2

4. Return control to TM1

5. Call commit at TM1

6. Proxy the commit to TM2

7. Prepare the resources at TM2

8. -- CRAAAASH AT TM2 --

Now, when TM2 recovers it wont have any record of the transaction at TM2 so the prepared resources at TM2 are orphaned forever. Hence TM2 needs to be able to be able to detect these orphans and roll them back (presumed abort).

Tom

[tomjenkinson]

Another way a subordinate can be orphaned is by one phase commit.

1. Enlist a single proxy resource at TM1

2. Propagate to TM2

3. Enlist two resources at TM2

4. Return to TM1

5. Commit TM1 thinks there is only one resource so does a one phase commit

6. Propagate to TM2

7. TM2 knows there are at least two resources here so prepares resources

8. Crash at TM2

The resources prepared at step 7 are now orphaned

[tomjenkinson]

Not being able to sleep, I was thinking about the following. Another way an orphan can be left (this time a subordinate transaction rather than a subordinate transactions resource) is:

1. Start transaction at TM1

2. Propagate to TM2

3. Enlist the database at TM2

3. Return to TM1 but before the proxy resource can be enlisted at TM1....

4. -- CRASH --

This scenario has a couple of issues salient to us.

1. The transport is going to need to keep a record *before it talks to the remote server* of all remote servers it talks to so that it can generate proxy XAResourceRecoveries to make sure when the server housing it crashed it did not leave any transactions in flight.

2. The case provides the absolute requirement of storing the parents node name in an Xid as during recovery this is the only information we have to help identify which subordinate transactions in flight are owned by a recovering parent

[marklittle]

I'd be more concerned about doing this with the node name than with the EIS, since the former is not as dynamic as the latter, i.e., I suspect the EIS data is going to be almost read only whereas the node names will be added to almost constantly. That'd make a central repository or cache for node names far less useful in a failure scenario than the equivalent for EIS.

[tomjenkinson]

Hi guys,

I just want to prefix this answer with the statement that given unlimited data length I don't mind whether or not node names were encoded as Strings, Ints, BLOBs or GLOBs.

The administration of this thing is one thing. But in terms of the XID structure it only has 64 bytes to play with in the bqual.

The bqual needs the following:

1. Sequence number - we all agree with this

2. Parent node name - This is vital for recovering orphan subordinate transactions as explained above

3. Subordinate node name - This is vital for recovering orphan subordinate resources as explained above

4. EIS name - required for debug

Lets say we keyed the EIS name down to an int - 4 bytes - using Jonathans suggestion and the algorithm I indicated earlier, thereby paying the performance cost of reading/writing to a dynamic EIS key file and looking up the EIS name in the key map each time.

Next, the sequence number is 28 bytes because it is a Uid, we could look at trying to make this smaller but it needs to be unique within the scope of the gtrid.

That leaves us 32 bytes to encode the two node names (parent and subordinate). If someone can confirm that we can have a validator on CoreEnvironmentBean that says node names that can't be .getBytes("US-ASCII").length'd down to 12 bytes or less can be "throw new CoreEnvironmentBeanException"'d is fine, I can move back to Strings for these within an hour or so (I say 12 bytes so we can write the byte[] and its length - an extra 4 bytes, to be fair we could use a short here so call it 14 bytes as the max getBytes.length of the node name).

I would also need it confirming that we are fine to make the adjustment of the EIS name down to an int which is just logged to file and so admins debugging issues will need to crack open that file to reverse lookup the actual EIS name from the keys which are going to be encoded in the Xids.

Tom

[marklittle]

I know how long an XID struct is

[tomjenkinson]

Hi Mark - sorry, I know you do!! Just setting out the issue with node names being stored uncompressed, unless they can be encoded into 14 bytes is all. If the node name can't be fit into 14 bytes as a string then it is going to need to be indexed somewhere and the only thing I can think of is the central file with node names in.

[marklittle]

I'm joking with you

Can you compress the sequence number? What I mean by this is that if it's a Uid and based on the gtrid then maybe you can get rid of the IP address component of the sequence number because it's implicitly there in the gtrid anyway?

[tomjenkinson]

Yeah, possibly, I wonder how much space we are going to save, but yeah, I can take a look  at that. That said, what size should we be looking at aiming for allowing node names? At the moment, it is possible that 14 bytes is enough to encode the node name? David, what should be aiming to allow size wise for a remoting node name?

[marklittle]

Well if you use the node name bytes as bits and assuming a number is sufficient for the name and not a string, then you've got 112 bits to work with which allows you to encode about 5.2E33 numbers. I doubt anyone will have that many nodes. So maybe there's scope there too?

[tomjenkinson]

I doubt that they would have more than that too - although it would make an interesting case study

But I also assumed that no one would have more unique nodes than would fit in a standard int either. I think the problem with encoding "node names" as numbers is that David is saying he doesn't think people will want to administer the nodes as numbers, rather he preferes a string derived from the hostname and domain (out of interest David how do you propose to deal with multiple nodes on the same machine, plus remember that for TS these node names are sticky per instance you want to recover).

[marklittle]

Aren't we mixing two different issues here? How we represent the node name within TS (within the XID structure) and how that node name is presented to users/administrators? Our top priority has to be data consistency in the presence of failures. If that means that a sys admin either can't do something for a while (because, say, a centralised name server that maps int node names to a nice string is unavailable) or has to work with ints, then I'm prepared to consider that any day over the alternative.

[tomjenkinson]

Mark Little wrote:

Aren't we mixing two different issues here? How we represent the node name within TS (within the XID structure) and how that node name is presented to users/administrators? Our top priority has to be data consistency in the presence of failures. If that means that a sys admin either can't do something for a while (because, say, a centralised name server that maps int node names to a nice string is unavailable) or has to work with ints, then I'm prepared to consider that any day over the alternative.

+1

All I want is for nodeIdentifiers to be provided to TS through the CoreEnvironmentBean as an integer and that the same recoverable instance of TS gets the same integer node name each time, I can then happily encode this in the XID and failure recovery etc proceeds as normal. What systems are used to provide that node identifier to TS should be largely a black box to me.

[tomjenkinson]

Mark Little wrote:

Can you compress the sequence number? What I mean by this is that if it's a Uid and based on the gtrid then maybe you can get rid of the IP address component of the sequence number because it's implicitly there in the gtrid anyway?

Sorry I didn't get chance to respond to this yesterday. I have created a BranchId class which is a clone of UID but doesn't utilise the hostname part of it. I have run a build now to check it hasn't broken anything and it is fine but I haven't put this back until we know the route we are going to take.

Removing the hostname from BranchId and giving EIS names transient integer representations gives us a space of:

64 (bqual)

     - 12 (branch id)

     - 4 - eisname key

-------

EDIT

     =  48 bytes (available for node names)

That means there are up-to 24 bytes per name, we will need to store the name length (which given the size available need only be a short) so less 2 bytes leaves 22 bytes to store a node name.

----------

If this is enough space (and Mark and Jonathan agree that this change to branch qualifiers is sound - it will affect all Xids)  then I can have this changed within an hour and we can proceed to get this out into AS7 trunk. I will have to change the EIS name to an int too of course, but I can add a routine that converts EIS names to the next available int (or short if you shout up now) and logs out the conversion values to the log file as a warning. Would this map of EIS names need to be persisted between runs? I assume not as the transactions jboss logging output file will contain the id of the EIS against the provided name for that particular run. If we *do* need to persist this mapping file then we will need to have a think about the best place to put it, can we assume to dump it out into the current working directory?

The alternative is to keep with ints for node name, and I can put this out there right now with no change required to EIS name.

Either way I would like to get some agreement as this discussion hasn't slowed the process down so far as I have been comprehensively testing this functionality relying purely on the availablity of the node name "tokens" in an Xid but as I near the end of available tests I can perform, it really needs to get into Davids hands to provide this new functionality into AS7.

[marklittle]

Let's get a response from David before proceeding, but I continue to believe that a number is fine as long as we can provide some higher level tool to isolate users from this.

BTW, if it's 27 bytes then does that mean it could (almost) be a Uid?