Data inconsistency for XA transaction when db resource returns XAER_RMERR code for lost connection

[ochaloup]

Hi,

I would like start discussion about scenario which ends with inconsistent data for XA transaction when XA resource returns XAException.XAER_RMERR and continues with commit.

This behaviour is against specification as it mandates that:

An error occurred in committing the work performed on behalf of the transaction

branch and the branch’s work has been rolled back. Note that returning this error

signals a catastrophic event to a transaction manager since other resource

managers may successfully commit their work on behalf of this branch. This error

should be returned only when a resource manager concludes that it can never

commit the branch and that it cannot hold the branch’s resources in a prepared

state. Otherwise, [XA_RETRY] should be returned.

see discussion and Tom's comment at Bug 1169671 – Recovery scenario where db connection is halted after prepare phase does not rollback resource

but the problem is that a lot of databases behaves in this way. Databases as PostgreSQL, MSSQL or Sybase throws XAException.XAER_RMERR anytime when connection is lost.

Narayana transaction manager then rollbacks the rest of the transaction. If we have following scenario the result is inconsistent data.

1. prepare DB xa resource
2. prepare second xa resource
3. commit DB xa resource
4. DB commits
5. connection crashes (before confirmation is received by transaction manager)
6. jdbc driver returns XAException as connection is down

There are now 2 cases. At least for databases that EAP app server supports.

The jdbc driver returns XAException.XAER_RMFAIL or XAER_RETRY. That's ok as all the subsequent xa resources are committed. Apart from a small issue of recovery manager that will repeat a try to commit non-existent XID (as DB already commits). This should be fixed by [JBTM-860] use XAResourceWrapper metadata for assume complete - JBoss Issue Tracker.

The second case is the problematic one.

The jdbc driver returns XAException.XAER_RMERR. In this case DB commits but after connection is lost method doAbort is called for the rest of xa resources. Thus the other resources are rollbacked.

I understand that it's problem of jdbc driver and incorrect error code but it disconcerts me a bit the fact that databases like mssql, postgresql etc. could end up with inconsistent data. At least for this (corner) case.

Is this just a documentation issue from TM point of view? Or could Narayana somelike prevent that situation?

Thanks

Ondra

[tomjenkinson]

Thanks for raising the discussion more widely Ondra. My feeling is that for the data inconsistency we must remain consistent with the spec and treat it as a rollback as it is possible if we treat it as anything else it could result in unexpected data inconsistencies with products that correctly implement the XA API in this area. I am happy to discuss this though if others can propose a safe alternative?

Thanks again,

Tom

[ochaloup]

It's worthy to say that the issue with data inconsistency does not occur for example for Oracle database as its jdbc driver returns XAER_RMFAIL when connection is lost. This comes under the the case 1 where all resources are committed.

There was created a issue for PostgreSQL jdbc driver which asks for changing the error code to follow the spec.

Incorrect return code for error on call of commit during 2PC · Issue #236 · pgjdbc/pgjdbc · GitHub

[tomjenkinson]

Thanks for the clarification, I wonder if you should forward your github issue link to http://www.postgresql.org/support/submitbug/. It's strange it hasn't been commented on yet...

[ochaloup]

I've already sent info to pgsql-jdbc list as I was asked at [1]. But I'm not able to find out the archive of the pgsql-jdbc mailing list to put the reference here. Then it was the motivation to open github issue afterwards.

[1]

http://www.postgresql.org/message-id/[e-mail]

[marklittle]

I agree - it should be fixed in Postgres.