[ochaloup]

tomjenkinson  wrote:

Hi Ondra, delistResource is just a way to register the resource from the transaction so you can do work outside the transaction - it doesn't actually leave the transaction.

ah, ok, I see. Sorry for my misinterpretation and thank you for the kind correction.

[mmusgrov]

REST-AT provides a model that is the same as XA 2PC but using REST endpoints instead of programmatic APIs: in other words there is no leave operation.

The LRA (Long Running Action) specification is provisional but it does allow a participant to leave an LRA at any time prior to the coordinator receiving a request to cancel or close the LRA. However, I don't think you would want to follow the LRA model for the XA enhancement being discussed but the proposed "leave(XAResource)" seems reasonable.

As an aside, is Leading the development of open, vendor-neutral IT standards and certifications | The Open Group still accepting updates to X/Open specifications?

[tomjenkinson]

I might be missing something but I think REST-AT has a leave operation described here: Narayana Project Documentation

Perhaps that is just in our implementation and not in the spec?

[tomjenkinson]

I think it is in the spec. Line 489 of http://narayana.io//docs/specs/restat-v2-draft-8-2013-jul-29.pdf  (it doesn't term it leave there but does refer to a read only optional behaviour.

[mmusgrov]

You are right, it is kind of modelled on OTS where the enlist operation returns a recovery coordinator. The REST-AT spec similarly returns a recovery coordinator URI when a participant joins a transaction but supports an HTTP DELETE operation on the returned URI which is used to allow participants to leave the transaction. JTA does not have an equivalent though.

[tomjenkinson]

In order to add a "leave(XAResource)" operation to TransactionImple we do need to clarify when it can be called because XA not providing functionality to allow a resource to remove itself from participating in a transaction could lead to inconsistencies if we are not careful.

We would likely want to say that it is illegal for "leave(Xid)" to be called while a transaction branch is active in the resource and leave it to implementations to guarantee that.  My main question is whether is possible to identify a point where we can be certain this operation was called outside of a "start(Xid)/end(Xid)" pair but before "prepare(Xid)".  I am wondering if it is possible a resource could do that during a Synchronization::beforeCompletion call:

    transaction.leave(this);

If necessary we could add the ordering of the required sync into the WildFly TSR (wildfly/JCAOrderedLastSynchronizationList.java at master · wildfly/wildfly · GitHub ) to ensure it's beforeCompletion is called after any sync which might be responsible for delistResource (such as IronJacamar for JCA) so we know end(Xid) has been called. I realise it is not a problem for EJB as you your sync can call delistResource(XAResource), leave(XAResource) in the same thread but I was wondering about broader cases if it moved forward to the specification body.

[dmlloyd]

It seems a lot easier for the RM if they could just return a value from end() and let the TM take care of the hard stuff.

Remember that it's better to take the complexity to the TM than push it on the RM or app server.

[tomjenkinson]

xa_start and xa_end are related to transaction association and as Mike brought up earlier, I am not sure if the standards body who maintain the XA specification would be accepting changes at the moment. It was last updated in 1991 and so whether it would be reopened for an enhancement where a workaround to support that enhancement could be made in standards that are active would be something to ask the Open Group. Even then changing xa_end would add further behaviour into method which might not be desirable.

I can't really say it is adding too much complexity to add a new method to Transaction to allow the resource to leave.

To work around threading things, perhaps what we could do is call it "requestLeave(XAResource,LeaveCallback)"

If you call requestLeave when the resource is in the middle of being used (between xa_start and xa_end) and so can't be removed immediately we could call the LeaveCallback when it is not opportune to do so and notify you via the LeaveCallback::left() method.

[dmlloyd]

tomjenkinson  wrote:

xa_start and xa_end are related to transaction association and as Mike brought up earlier, I am not sure if the standards body who maintain the XA specification would be accepting changes at the moment. It was last updated in 1991 and so whether it would be reopened for an enhancement where a workaround to support that enhancement could be made in standards that are active would be something to ask the Open Group. Even then changing xa_end would add further behaviour into method which might not be desirable.

I don't see why it matters at all what the XA specification does or says about this.  JTA is only a mapping of XA, and that is arguably only a historical perspective.  The enhancement I described does not countermand XA in any way, nor does it require changes to XA, nor does it introduce any incompatibilities with XA.  We could certainly request an enhancement to XA but it's definitely not in any way necessary and so the discussion about the XAResource enhancement should therefore be completely separate.

JTA already differs from XA in several ways for XAResource alone (see JTA § 3.4)

tomjenkinson  wrote:

I can't really say it is adding too much complexity to add a new method to Transaction to allow the resource to leave.

To work around threading things, perhaps what we could do is call it "requestLeave(XAResource,LeaveCallback)"

If you call requestLeave when the resource is in the middle of being used (between xa_start and xa_end) and so can't be removed immediately we could call the LeaveCallback when it is not opportune to do so and notify you via the LeaveCallback::left() method.

This is clearly adding significant complexity.  My basic problem is this: it is the resource, and the resource alone, which potentially has knowledge about whether it has done any work during the transaction.  It is the transaction manager alone which has control over its commit processing.  Asking the resource, or a party working on behalf of a resource, to remove itself from the transaction at the opportune time is turning this process on its head in several ways (not to mention introducing a lot of space for error); the substantial complexity cost of this type of API seems clear.  It is far simpler and more logical for a resource to be able to answer the question of "have you done any committable work on behalf of this transaction".  The end method seems like an ideal time to ask and answer this question for both the TM and the RM.

[tomjenkinson]

I don't think it is adding too much complexity to the resource, the resource would not even need to keep track of it's association. We can just add a method (or maybe a pair of methods) it can call at any time:

leave(XAResource)

When a resource wants to leave it calls leave. I was thinking we could add a callback version if it was useful for the resource to know when it was removed (if we decide it can't be done synchronously) but maybe we determine there is no advantage to knowing that formally anyway so we could just keep it to the single leave(XAResource) method initially.

When leave is called we can either track requests to leave and remove them at the next branch thread disassociation opportunity (i.e. after the next xa_end) or we could maybe just remove from the transaction straight away. That was why I was thinking of calling it requestLeave as you may not be removed from the transaction until later if xa_start had been called on you but not xa_end.

This would be closely consistent with REST-AT: narayana/Transaction.java at master · jbosstm/narayana · GitHub - however that just takes the approach of removing the participant at once. Perhaps we should take that approach.

Providing a leave() method to be called by the resource is useful as it means that a resource can call it when it knows it wants to leave rather than waiting for xa_end to be called. E.g. in your case as soon as it detects a second XAResource already exists for the same server.

[dmlloyd]

tomjenkinson  wrote:

I don't think it is adding too much complexity to the resource, the resource would not even need to keep track of it's association. We can just add a method (or maybe a pair of methods) it can call at any time:

leave(XAResource)

When a resource wants to leave it calls leave.

This is one thing I'm having a difficult time understanding: why (generally speaking) would a resource know when it wants to leave?  It cannot predict what it will be used for, and when, or when (or even if) the transaction will eventually be prepared.  It is the transaction manager which drives completion, and the end user code (not the resource) which drives the transaction manager.  The resource simply has no knowledge of this, and rightly so.

In addition, it's generally difficult for a resource to know what Transaction it is a part of.  The resource manager knows the XID of the transaction it is associated with, but nothing within JTA (that I can find) guarantees that that transaction is in turn associated with the calling thread when XAResource methods are invoked.  The enlist and delist methods are on Transaction and don't specify a thread context, nor do the start and end methods indicate anything about the thread of control.  An XAResource simply cannot be assumed to have access to the Transaction, based on anything I can find in the specification.

tomjenkinson  wrote:

When leave is called we can either track requests to leave and remove them at the next branch thread disassociation opportunity (i.e. after the next xa_end) or we could maybe just remove from the transaction straight away. That was why I was thinking of calling it requestLeave as you may not be removed from the transaction until later if xa_start had been called on you but not xa_end.

This is an example of semantic complexity which can cause problems for a prospective new API.  The leave() method is adding several new logical states which are really not necessary to realize this kind of pre-prepare remove optimization.  And it is not hard to argue that JTA already has far, far too many logical states.

tomjenkinson  wrote:

This would be closely consistent with REST-AT: narayana/Transaction.java at master · jbosstm/narayana · GitHub - however that just takes the approach of removing the participant at once. Perhaps we should take that approach.

Providing a leave() method to be called by the resource is useful as it means that a resource can call it when it knows it wants to leave rather than waiting for xa_end to be called. E.g. in your case as soon as it detects a second XAResource already exists for the same server.

The leave approach does not seem optimal for this use case (removing an already-enlisted resource) either.

Adding the resource and then turning around and removing it again is a cumbersome and problematic approach.  If two threads add the resource at the same time, the resource would have to be the one to detect the race via TMJOIN.  The resource has to somehow communicate that information back to the container code which had access to the Transaction object and called enlist, which will be difficult due to the aforementioned association problem as well as due to the new states/races inherent in the XAResource calling back to the transaction.

The most common approach for concurrent systems to avoid duplicate associations is an add-if-not-already-added method (with a success or failure indication), coupled with a query-if-present method.  This keeps the container side and the resource side nicely separated.

Based on my experiences, I've come to this conclusion: in any case where communication of information which originates in the XAResource must occur, such communication should only be done via XAResource return values and exceptions.  Anything else is going to be unwieldy, or introduce too many new error states, or increase complexity too far.  Nothing in this discussion seems to provide evidence to the contrary.

[tomjenkinson]

Just to point out that this is what I would envisage your XAR looking like:

XAResource::end(int flags) {

  int result = doEnd(flags);

  transaction.leave(this);

  return result;

}

As I thought we discussed in your comment 6 it seemed reasonable?

Anyway, iIf I understood your use case correctly you will know after the very first call whether or not you are going to do work on this branch because you will know whether you are using the same subordinate or not.

But I can also understand how it is additive and some users may not be able to easy know if they are going to do read only until the last case. This is probably why XA has that state in the prepare call. I understand you are trying to optimize out the need for the RPC in your case and hence the looking for the change. mmusgrov is going to do some more research...

[dmlloyd]

tomjenkinson  wrote:

Just to point out that this is what I would envisage your XAR looking like:

XAResource::end(int flags) {

  int result = doEnd(flags);

  transaction.leave(this);

  return result;

}

But where does "transaction" come from?  The XAResource doesn't have it and can't really get it.

tomjenkinson  wrote:

As I thought we discussed in your comment 6 it seemed reasonable?

It works from the narrow point of view of WFTC, which controls the transaction from the "outside" and thus allows our XAResource to have more knowledge than it otherwise would have had; because of this we're already in a good position to utilize this approach.  However, it's not the best approach from a general XAResource implementation point of view.

[tomjenkinson]

dmlloyd  wrote:

tomjenkinson   wrote:

Just to point out that this is what I would envisage your XAR looking like:

XAResource::end(int flags) {

  int result = doEnd(flags);

  transaction.leave(this);

  return result;

}

But where does "transaction" come from?  The XAResource doesn't have it and can't really get it.

XA supports communication from a DTP RM to a TM via the ax_* methods so there is precedence for communication initiated by the resource manager to the transaction manager. I was thinking therefore the analogy for Java would be an injected/looked up transactionmanager?

To support that, from my reading of XA I think xa_end needs to be called from the specific thread where the transaction is active:

"The xa_start() and xa_end() routines pass an XID to an RM to associate or dissociate the calling thread with a branch. The association is not necessarily the thread’s initial association with the branch; its dissociation is not necessarily the final one."

But let's see what Mike or others comes back too with as they may have a different take.

[tomjenkinson]

Just to say that Mike (via myself) sent the query to the JTA spec mailing list: Possibility of a 2PC optimisation?