JBossDeveloper

Hi,

The implementation for this issue (registered as https://issues.jboss.org/browse/JBTM-916) has been put back to svn.

The implementation I went for structures the XID as follows:

int formatId = // new sequential number

byte[] globalTransactionId = // Uid + Node name of transaction originator

byte[] branchQualifier = // Uid + Node name of subordinate + Node name of subordinates parent + EIS name

In order to fit the node names in the space available they have had to be encoded as integers (rather than strings as they used to be). I appreciate from Davids points above that this will add configuration complexity as nodes must be assigned a unique transaction manager identifier and remoting string but at the moment I can't really see any other way around this (I will elaborate further below on why the data is required).

I am happy for the node names to be changed back to Strings, but we have a space issue then which would definitely need addressing.

The space we are most constrained by is the bqual, it must contain a Uid is 28 bytes (this guarantees we have a globally unique identifier for any particular branch), it also needs two node identifiers which I opted for using integers (i.e. 2 x 4) , and an EIS name that can consume the remaining part of the bqual and is typically the JNDI name of the resource which can be up to 28 bytes. If there was someway to compress three reasonable length strings into the bqual (or put another way we can be certain that two string node names and an EIS resource name can be encoded into javax.transaction.xa.Xid.MAXBQUALSIZE - com.arjuna.ats.arjuna.common.Uid.UID_SIZE bytes [64 -28]) then it is a trivial change to revert node names back to Strings.

Now, onto the reasoning for why the data is required at each point, before this I will clarify one piece of terminology:

orphan resource = a resource that has prepared and stored an XID but where the transaction manager did not have chance to write out a record of the XAResource before it crashed

A> global transaction ID - This definitely needs a UID for uniqueness, it also needs the node name encoding in it so that orphaned (JTA) resources can be rolled back.

B> branch qualifier - Each RM needs a identifier unique within the global transaction ID, it also needs the subordinate node name to ensure the transport is able to filter out Xids returned from the XATerminator recover method so that different root transaction managers don't try to recover each others transactions. It also needs a subordinates parent node name for a similar reasoning to the node name encoded into the gtrid, i.e. to ensure that when a recovery manager for the suboridnate starts up it can detect XA resources of a subordinate transaction where the XAResource prepared and stored the Xid but the subordinate transaction itself failed to prepare.

In terms of the bqual, it is not an absolutely vital component to store the parent node name. If you look at  https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Final/atsintegration/tests/classes/com/arjuna/ats/jta/distributed/server/impl/ServerImpl.java -r 37583 you can see that as I have a single ProxyXAResource per Xid and not a single XAR for all Xids I am able to filter the Xids returned from XATerminator.recover on format ID and gtrid alone. This is because I know that per subordinate transaction there should only ever be one single proxy XA resource talking to it (see the design discussion I am about to create for more details on this). Providing the parent node name though allows you to create a proxy XA resource that can recover all XIDs for a subordinate in one go, rather than calling recover multiple times.

Hopefully this clarifies the motivation why the Xid has been encoded as it has and why I have had to move to integers for node names.

Tom

Hi Mark,

In terms of raw XID manipulation:

I have ensured that in the case where we have none-distributed JTA the cost of this is zero (well, an initial check when generating the first XID has to compare a boolean to see if this is a subordinate).

In the case of existing JCA code there are a couple of further boolean checks when resolving existing SubordinateTransactions to check if the inflowed XID formatID is equal to one of ours, if false, do old comparison, if true do new comparison (which actually scores a hit by comparing gtrid only so that should be somewhat faster than comparing gtrid and bqual).

So essentially, the current system should remain as performant as it was. The distributed JTA case I have not performance tested as it is new functionality with nothing to compare against, though as the node identifiers are now integers I suspect encoding an XID will be more efficient than encoding an older XID.

In terms of architecural principal:

My tests for this simulate distribution via classloader separation. When David is able to consume the TS work, we can see what the cost of the distributed JTA vs JTS is "in real life". That said, I have placed a restriction that the graph of proxy XA resources must not contain loops (working with this is demonstrated in my example). This will ensure that transaction completion is efficient as you don't loop around the same nodes during completion, e.g. a transaction that flows through nodes 1,2,3,3,1,3,1,3 would get automatically merged (as the transport knows not to add completion links for nodes that are already connected in this transaction) and therefore gets ordered as 1,2,3 (though unfortunately not 1,2 and 1,3 which would be more efficient, but as the link from 2,3 is created before the link from 1,3 there is nothing we can do).

Tom

Performance tests on the XID generation?

Sure, but like I mentioned above. I haven't really altered the XID creation too much. Basically, the old structure is:

XidImple(OLD)

int formatId

byte[] gtrid { UID + nodeName:String }

byte[] bqual { UID + eisName:String }

XidImple(NEW)

int formatId

byte[] gtrid { UID + nodeName:int }

byte[] bqual { UID + nodeName:int + nodeName:int + eisName:String}

Generating the two should be fairly similar shouldn't it? Or maybe I am missing something? To create a harness to test this would require both XidImple to be present (which as the new one replaced the old one would mean two builds of TS).

Same here

Unfortunately I am sure after having several discussions with Jonathan about this (though happy to be disproved).

What we absolutely must be able to do is allow the recovery manager to detect orphan resources.

Typically we detect the orphan by looking in the gtrid for the node name (recall it is UID + node name). And that is OK now and moving forward for the none distributed case.

In the distributed case we are adding though, when the recovery manager is looking for orphaned resources registered against the *subordinate* it can't use the node name from the gtrid as that is the node name of the root TM rather than this subordinate.

Therefore I have put the node name of the subordinate in the bqual (which the subordinate transactions can edit).

Unfortunately a bqual can only be 64 bytes. In this 64 bytes we have to get:

1. The Uid (this could have been your path solution catted with a sequence number but I went with Uid to minimise change but they are the same for this discussion as they are only part of a bqual) - a Uid is 28 bytes constant

2. The "EIS name" - as I mentioned this is a requirement from customers - a variable length string

3. The subordinate node name (so crash recovery can detect orphan XA resources for the subordinate as described above)

4. The parent node name (this was added so the transport can choose to be implemented by a single proxy XA resource per local server that can recover for multiple remote servers and therefore needs to filter out the Xids it gets back from the call to XATerminator::recover by parent node name rather than Xid) *potentially this can be removed - the example I created https://svn.jboss.org/repos/labs/labs/jbosstm/branches/JBOSSTS_4_15_0_Final/atsintegration/examples doesn't actually use this as I chose to create a new proxy per transaction, performance testing of the transport will indicate which is the more efficient*

So at a minimum we need to get (i.e. if we confirmed that the parent node name would not be of use):

1. The unique number for the branch

2. The EIS name

3. A node name

Into the 64 bytes of the bqual

@Mark great point, though we could actually do the same for the node name couldn't we I suppose? And that would mean that we wouldn't be impacting our current code for EIS name in favour of this new functionality

The AS can have a file that says

"David's"   prefered name  TS node name

-----------------------------------   --------------------

"foo"                                   1

"bar"                                   2

"baz"                                   3

Basically it works a bit like a bespoke form of hashing where the hash of the remoting identifier is manually specified in some file, as you suggested above basically but for node names rather than EIS names.

To clarify, if we did knock the EIS name down to an int/short as Mark indicated was feasible. We would have the following:

byte[]:64:bqual   {Uid:28:sequenceId,String:variableLength:subordinateNodeName,String:variableLength:parentNodeName,int/short:4/2:EISName}

Meaning we would still need to be able to get two strings into the remaining 34/32 bytes, another reason I suggested converting the node name in the AS down to an int as that saves the space of two strings rather than just one.

Plus the API for getting the EIS name is already out there and is being used (I assume). We would therefore need to get folks to change their implementation of: org.jboss.tm.XAResourceWrapper::getJndiName to return this integer key for the EIS name table (or pay the cost of looking up this String in the EIS String to int table each time and XID is created).

Either approach (keying node names or keying EIS names) levies an administration overhead but for space considerations and the fact that the EIS name API is already out there I think we should use ints in the XID for nodes rather than EIS names - again though, happy to be dissuaded.