One initial observation, is it important to you that Synchronization::afterCompletion(int Status) may not be called in the case of failure? If so you are likely going to need to register your own XAResource with the transaction as that is the only way to be guaranteed a callback.

No, that's ok. We only need to listen to successful transactions, and don't have resources which could need cleanup depending on this, so it's ok if some synchronizations are simply lost without a callback.

Also, you can add your own record type to Narayana to record data instead of an XAR.

That's very interesting. I guess it would require some cross-teams cooperation to get such an extension hooked up inside WildFly, depending which capabilities get activated for a specific user deployment.. I'll keep it in mind but I'd need to solve the ordering puzzler first for any of this to be of use.

A further issue you might come across is that the way the update is applies is (possibly) not stricly guaranteed by the resource manager (at least not that I am aware).

 

Consider a TX with multiple XAR registered. It is possible that the RM can go down after prepare with multiple TX active. That means that the recovery manager will take over. The failure could be transient and further updates in a TX then do apply successfully. By that way update 2 (for example) may be committed before the recovery manager is able to sucessfully commit update 1.

 

Would that be an issue?

Our goal is to stay in synch with the relational database, so I guess we're open to such issues but then that's business as usual, and people need to rely on the recovery manager exactly to restore correctness to the world.. right? So I guess it's ok to accept that we might be out of synch as long as partial failures need to be fixed. What we would need is to make sure that when the changes are finally applied, that these are also sent to our log (if they haven't already).

Consider that what I'm exploring is a performance optimisation for the happy path: I'm assuming we could detect that such a recovery was necessary, then trigger our less efficient alternative to repair from exceptional cases in which there's ambiguity about ordering. Thanks a lot for highlighting this case though! I definitely need to keep that in mind.

My first and foremost question is about the ordering of events though. I don't need Total Order in this binary log but at least changesets which share some common entries should not be reordered. Hibernate could do some magic but it's constrained to a single JVM (and would probably be very complex to do correctly), while the database should be able to see the full picture as it has to resolve such conflicts via some commitment ordering.

Looks like I'd need to access the database log, wondering if there's any helpful metadata accessible to our code. I'm guessing there isn't and I should look at an alternative approach?

Would the transaction ids represent a usable ordering criteria in case the isolation level was set to Serializable?

Hi Mike,

thanks for the interesting paper!

But you're right, using the creation time isn't suited. I actually would prefer to avoid using timestamps at all as it pushes a lot more complexity on the infrastructure like clock synchronizations hell.

The commit timestamp could be a solution as long as it was delivered by the database, and the RDBMS was configured for serializable transactions, but I'm understanding from other answers (and a chat with Jonathan) that such details just aren't available.

Emmanuel: yes that's probably a wiser direction, although it would require versioned entities.

I'm thinking that we only need this capability to "Sort" messages when there is a possibility that they might have been generated in a very close sequence of transactions which modify the same entities. But we can detect if there is a risk of ambiguity by looking ahead in the log:

We should be able to process chunks of the log stream and inspect if there's any possibility for potentially problematic event reorderings, and only if there is in fact some critical entity in this case we could resort to a new read from the RDBMS for this to act as final decision maker about what the correct state shall be.

This implies that sometimes we'll have to re-read the same entity graph again from the database, but I suspect that in practice that would be a small minority of all events being processed, so to provide the correctness I'm looking for but without a large overhead as we would have as by re-reading every single entity graph in the indexing backend.

With such a design using the entity versions would be a further optimisation, as people who would have a problem with excess re-reads because of some specific data access pattern they would have the option to enable versioning to improve indexing performance.. this would make the use of versioning an optional benefit, which I like more than it being a requirement for correctness.

Thanks all for your thoughts!