[mmusgrov]

I would like to add a note here to point out that the 1PC optimisation is not applied to subordinates [JBTM-2916] Disable dynamic1PC for subordinate transactions - JBoss Issue Tracker.  If we were to apply the optimisation to subordinates then the subordinate could commit during the prepare phase [1] but if the final outcome of the top level transaction is rollback then we end up with heuristics.

I have added this comment because it reduces the performance gain from adding the proposed enhancement.

[1] Note that the reason we commit the resource during prepare rather than first in the list at the commit phase is to avoid the log write.

[tomjenkinson]

Just FYI the issue was reported here: - JBoss Issue Tracker

[ochaloup]

Hi mmusgrov,

I have trouble to understand exact meaning of your comment. Would you be so kind and elaborate a bit?

My hesitation is that I do not understand exact trouble at [JBTM-2916] Disable dynamic1PC for subordinate transactions - JBoss Issue Tracker .

Let's say we have one subordinate resource which belongs to a top level transaction. Is there permitted the 1PC? I can image if we have subordinate transaction which consists from work on 2 resources then the top level calls commit subordinate txn (instead of prepare as 1PC could be applied). The first resource in the subordinate txn commits and second errors with rollback. This is the point of possible inconsistency?

What if I have two participant of top level txn. One is subordinate one is db xa resource. If subordinate txn returns xa read only on the prepare, the db xa resource is fine to be 1PC commited, right?

What are the reasons behind the 1PC restrictions?

Many thanks!

Ondra

[tomjenkinson]

Hi Ondra,

It is just the subordinate transactions themselves that can't do the Narayana 1PC optimization during prepare phase. A top level transaction with subordinates participating in it should be OK.

Consider:

Root -> Subordinate1 -> XAR1

                                    -> XAR2

         -> Subordinate2 -> XAR3

You don't want Subordinate2 to do a 1PC optimization on XAR3 during prepare phase.

Hope that helps,

Tom

[ochaloup]

Ok, I see that now. Thanks tomjenkinson !

[tomjenkinson]

No problem!

[jhalliday]

Trying to understand the requirements here, in preparation for reviewing the patch...

It seems to me you are taking the position that it's the logging that is expensive here, rather than the network round trips. Hence reducing the number of Resources (i.e. round trips) is not a priority, but using 1PC is?

Some RMs, notably Oracle, will optimize the case where multiple resources (branches) for the same transaction are operating on the same RM. In such case, the RM will ensure that N-1 of the resources return RDONLY, whilst one returns OK if needed. That allows TM logging to be elided where possible, though it still requires a separate commit call for one resource. It's not specified which resource returns OK, but it's clearly possible to implement the optimization such that the last one invoked does so, in which case the TM can do 1PC where possible. To achieve this, the RM needs to track the number of branches (distinct bqual) for each transaction (gtrid) by looking at the Xid provided on the call. Is there some reason that's not possible with Wildfly as the RM?

There are different points in the transaction lifecycle where end can be called on the resource. Would it be useful to enhance the semantics such that a) the return value can indicate if the RM expects a subsequent prepare/rollback in the RDONLY case, or if resource cleanup will be automatic? and b) the TM can signal the RM that the call is directly pre-commit and the RM can actually bundle the prepare stage, e.g. endAndPrepare rather than endWithResult.

[dmlloyd]

jhalliday  wrote:

Trying to understand the requirements here, in preparation for reviewing the patch...

It seems to me you are taking the position that it's the logging that is expensive here, rather than the network round trips. Hence reducing the number of Resources (i.e. round trips) is not a priority, but using 1PC is?

Really both are expensive (for some value of "expensive"), though nothing in the proposed enhancement would impact the network round trips (it's not a trade-off in this case).

jhalliday  wrote:

Some RMs, notably Oracle, will optimize the case where multiple resources (branches) for the same transaction are operating on the same RM. In such case, the RM will ensure that N-1 of the resources return RDONLY, whilst one returns OK if needed. That allows TM logging to be elided where possible, though it still requires a separate commit call for one resource. It's not specified which resource returns OK, but it's clearly possible to implement the optimization such that the last one invoked does so, in which case the TM can do 1PC where possible. To achieve this, the RM needs to track the number of branches (distinct bqual) for each transaction (gtrid) by looking at the Xid provided on the call. Is there some reason that's not possible with Wildfly as the RM?

This is a clever idea and I may try it out.  It does not seem to help the case where you have more than one RM though.

jhalliday  wrote:

There are different points in the transaction lifecycle where end can be called on the resource. Would it be useful to enhance the semantics such that a) the return value can indicate if the RM expects a subsequent prepare/rollback in the RDONLY case, or if resource cleanup will be automatic? and b) the TM can signal the RM that the call is directly pre-commit and the RM can actually bundle the prepare stage, e.g. endAndPrepare rather than endWithResult.

I think that could be reasonable from the RM perspective, but I'd be concerned that the TM might not then be able to do 1PC in some situations.

[jhalliday]

> It does not seem to help the case where you have more than one RM though.

Right. However, my initial thinking is this: the patch for doing this in the TM is intrusive and touches some old, finickety and critical code. If we can get most of the value without patching the TM that may be the safer bet. Do you have a feel for what proportion of use cases are impacted?

> I'd be concerned that the TM might not then be able to do 1PC in some situations.

Comes back to the round-trips vs. 1PC argument I guess - you save the extra round trips but don't get the 1PC. However, if you allow the resource to return 'done end. need separate prepare' as an option from endAndPrepare then the RM can signal that it's optimized to prefer extra round trips rather than forcing 2PC. If, in the same way that the last instance is the one to return OK, we make the first instance per-RM the one to return 'need separate prepare' from the endAndPrepare phase, we can potentially hint to other resources later in the list that they will be in a 2PC situation and should just prepare rather than returning 'need separate prepare'.

[dmlloyd]

jhalliday  wrote:

> It does not seem to help the case where you have more than one RM though.

Right. However, my initial thinking is this: the patch for doing this in the TM is intrusive and touches some old, finickety and critical code. If we can get most of the value without patching the TM that may be the safer bet. Do you have a feel for what proportion of use cases are impacted?

I'm not really sure TBH.  It is certainly valid to mix up (for example) JMS and database access with a subordinate EJB call though, so I think that the uses are there.  I may still pursue the optimization though just because it won't hurt anything.

jhalliday  wrote:

> It does not seem to help the case where you have more than one RM though.

Right. However, my initial thinking is this: the patch for doing this in the TM is intrusive and touches some old, finickety and critical code. If we can get most of the value without patching the TM that may be the safer bet. Do you have a feel for what proportion of use cases are impacted?

> I'd be concerned that the TM might not then be able to do 1PC in some situations.

Comes back to the round-trips vs. 1PC argument I guess - you save the extra round trips but don't get the 1PC. However, if you allow the resource to return 'done end. need separate prepare' as an option from endAndPrepare then the RM can signal that it's optimized to prefer extra round trips rather than forcing 2PC. If, in the same way that the last instance is the one to return OK, we make the first instance per-RM the one to return 'need separate prepare' from the endAndPrepare phase, we can potentially hint to other resources later in the list that they will be in a 2PC situation and should just prepare rather than returning 'need separate prepare'.

OK I think I understand what you're saying.  We do not however make a round-trip on 'end' calls, so it doesn't cost us anything to return the current known status of the transaction.  We do however need a round trip to prepare.  So in this regard, having an endAndPrepare would probably not make a difference to us compared to endWithResult since either way the only round trip will be to prepare.

[jhalliday]

hmm. So the wire protocol is such that the client side (i.e. TM side) knows the prepare outcome is RDONLY without asking the server? If not then endWithResult is actually counter-productive, since it forces end to be a remote call just to get the status. Multiplied by however many resources the server has, unless the client side can share the returned info between all enlisted instances for the given tx.

[dmlloyd]

jhalliday  wrote:

hmm. So the wire protocol is such that the client side (i.e. TM side) knows the prepare outcome is RDONLY without asking the server? If not then endWithResult is actually counter-productive, since it forces end to be a remote call just to get the status. Multiplied by however many resources the server has, unless the client side can share the returned info between all enlisted instances for the given tx.

No, endWithResult would not force a remote call.  The TM side knows that the prepare outcome will likely be RDONLY based on its past interactions with the subordinate, and that's what it would base the return value of endWithResult on.

[jhalliday]

Some thoughts now that I've had a bit of time to think it over:

The current API contract is unclear. I think what the proposed patch does is avoid calling prepare for records that have returned read-only at pre-prepare. That's wrong, or at least problematic, for resources that expect to do cleanup in prepare even for the read-only case. IMO pre-prepare is really about providing call order guidance. With that in mind, how does this play with the resource ordering problem discussed in support explicit ordering of commits for XAResources enlisted in a transaction · Issue #4 · javaee/jta-spec · GitHub  ? It seems to me that whereas we previously considered resource ordering based on type (e.g. put the JMS after the DB) what we're talking about here is based on instance state (read-only vs dirty).  I'd be happier feeding the requirement into the spec discussion and waiting to see what comes out, otherwise we face having to redo things if the spec moves in a different direction.

[dmlloyd]

jhalliday  wrote:

Some thoughts now that I've had a bit of time to think it over:

The current API contract is unclear. I think what the proposed patch does is avoid calling prepare for records that have returned read-only at pre-prepare. That's wrong, or at least problematic, for resources that expect to do cleanup in prepare even for the read-only case. IMO pre-prepare is really about providing call order guidance.

This is exactly what my original "endWithResult" proposal is all about: it simply hints about the order in which resources should prepare, and the hint can safely be ignored by the TM.

jhalliday  wrote:

Some thoughts now that I've had a bit of time to think it over:

With that in mind, how does this play with the resource ordering problem discussed in support explicit ordering of commits for XAResources enlisted in a transaction · Issue #4 · javaee/jta-spec · GitHub  ? It seems to me that whereas we previously considered resource ordering based on type (e.g. put the JMS after the DB) what we're talking about here is based on instance state (read-only vs dirty).  I'd be happier feeding the requirement into the spec discussion and waiting to see what comes out, otherwise we face having to redo things if the spec moves in a different direction.

The original issue is 6 years old, and furthermore there is, as of let, very little non-Red Hat specification activity on JTA.  Do you really think there's a chance that this could become a factor within, say, the next five years?

[jhalliday]

Well once the dust settles on the Jakarta EE transition I'm hopeful things will be able to move along at a less slothful pace, but yes, we may be able to crank out a proprietary solution more quickly than waiting on the spec.

In general I'm leaning towards a design decoupling the ordering hint from the lifecycle transition. A giveMeAnOrderingHint method should be called by the TM on the resource, at a point in time between the final 'end' and the 'prepare', but not mutate the resource state the way either of those do. The resource implementation may use 'end' to populate its client-side state cache so it can answer the giveMeAnOrderingHint question efficiently, but that's an implementation detail. The downside is that this doesn't provide for grouping end+prepare operations to save an additional round trip, but it's got a lot less opportunity for messing up the state lifecycle management and error handling that has traditionally been a thorny area for the TM.