Uses for HeuristicInformation

[mmusgrov]

We have an interface called com.arjuna.ats.arjuna.coordinator.HeuristicInformation which provides information about what kind of heuristic a transaction participant generated and a method to return a reference to an object that can be used to attempt resolution of the heuristic. The interface was aimed at users who want to provide their own AbstractRecord implementations and their own tooling to check and resolve the heuristic. When available the AbstractRecord.value() method is meant to return the HeuristicInformation.

We do not currently use the interface but I would like to know if anyone out there is using it and/or has a use case for it. The feedback will be useful for us to decide if it is worth adding to our tooling or whether it is safe to deprecate.

[marklittle]

Mike, why don't you add the use case as we've been discussing it so far? Would also be useful to include the interface definition for completeness.

[mmusgrov]

The full interface is:

package com.arjuna.ats.arjuna.coordinator;
/**
 * An implementation of this interface should be returned by
 * an AbstractRecord if it has caused a heuristic decision.
 */
public interface HeuristicInformation
{
    /**
     * The type of heuristic.
     * @return the heuristic type
     */
    public int getHeuristicType();

    /**
     * A reference to the entity that caused the heuristic.
     * @return the entity reference
     */
    public Object getEntityReference();
}

Use cases:

a. Exposing a third-parties representation of a heuristic type as defined in their AbstractRecord.

b. In terms of providing an interface method "Object getEntityReference()". Its difficult to see a use case for this one. I think we would need to return something other than Object from the method in order to do something meaningful within our tooling. We could change the method to:

HeuristicManagement getEntityReference()

Where a new HeuristicManagement interface then has a single new method:

resolve()

By default that would move the record back to the prepared list and allow us to commit it again, as our tooling currently allows. However, the advantage of providing an implementation of this new method is that the AbstractRecord implementer gets a hook to do something with his server side resource manager to help clear any transient error conditions causing the heuristic.

[mmusgrov]

Marks response to use case b) was:

Resolving a heuristic doesn't mean moving it back to the prepared state and replaying the transaction. That would be one possible option, but in most cases it simply isn't possible - the data has moved on and trying to revert it to the state that existed prior to the transaction beginning could result in cascading aborts, if possible at all. If you check the forget method in the OTS spec, it is simply meant to indicate that the resource can forget about the transaction. We can't call forget until we're told it's safe to do so. The idea with the original interface was that a tool (obviously not the tool we had) could call some operation on the reference returned under instruction by a sys admin and that would be one way to resolve the heuristic somehow - how is not important, it's implementation specific. Obviously the sys admin could use some other mechanism to achieve the same result, but the ultimate aim of the tool was that once we know the heuristic has been managed by the resource (somehow, e.g., by calling forget() if it's a JTS Resource) we can then remove that entry from the transaction log.

[tomjenkinson]

Sounds like the resolve() method needs to be a boolean in order to know whether we can remove the entry from completion events. It sounds like this method could possibly be called by our recovery manager, as well as or instead of the tooling.

public boolean resolve()

When Mike and I discussed this f2f. He did point out that there are some heuristics that can be resolved by moving back to prepared. For example if the RM is prepared, crashed and brought back up in read-only mode, when the DB is restarted again in write-mode it would allow a heuristic of this type to clear using the current tooling.

[marklittle]

boolean return or exception on void would be alternatives.

Yes, some heuristics could be resolved by doing some resource-specific work and moving the participant back to the prepared state. I did say that. It's just that we cannot assume that is the default or majority case, which appeared to be the assumption in the original part of our discussion. If we get HeuristicRollback or HeuristicCommit, the work's been done (or undone). There's nothing we can do. In fact there may not be much the sys admin can do without understanding what operations or business logic were being performed within the transaction(s) at the time. A real world example of a HeuristicCommit is dispensing money from a cashpoint before the account state has been validated. To resolve this requires a number of things to happen, including maybe charging an overdraft fee, chasing the account holder through the courts etc. The resource in this case never goes back to the prepared state.

[mmusgrov]

Mark Little wrote:

boolean return or exception on void would be alternatives.

Yes, some heuristics could be resolved by doing some resource-specific work and moving the participant back to the prepared state. I did say that. It's just that we cannot assume that is the default or majority case, which appeared to be the assumption in the original part of our discussion. If we get HeuristicRollback or HeuristicCommit, the work's been done (or undone). There's nothing we can do. In fact there may not be much the sys admin can do without understanding what operations or business logic were being performed within the transaction(s) at the time. A real world example of a HeuristicCommit is dispensing money from a cashpoint before the account state has been validated. To resolve this requires a number of things to happen, including maybe charging an overdraft fee, chasing the account holder through the courts etc. The resource in this case never goes back to the prepared state.

Good points. If the heuristic is in any of those states then we should disallow the recover operation in the tooling - I will add a JIRA for that.

We currently have two methods for resolving heuristics:

1. Move the record back onto the prepared list;
2. The admin performs the resolution in two stages
   1. uses the specific Resource Manager product tooling to resolve it at the product end;
   2. uses our tooling to remove the log entry - all of our log record MBeans contain a remove method: OSEntryBeanMBean (Narayana: narayana-full 5.0.4.Final-SNAPSHOT API)

A question I would like to raise is is this sufficient for resolving all types of heuristic?

If it isn't sufficient then this extra resolve method we are discussing should be used as a hook for a third party to plug in his own method of resolving the heuristic (including using our tool to remove the record and his own API to his RM to clear it).

[marklittle]

OK, so we're agreed that method 1 isn't sufficient and we need to make some changes to ensure that something which shouldn't be put back on to the prepared list isn't.

Method 2 works if there is a specific product tool for resolving the resolution. I suppose we could always assume that is the case until shown otherwise and I can't see a reason why it wouldn't work for all resources otherwise. I suppose the only downside is someone having to jump between two different tools.

[tomjenkinson]

I think that as we all agree that the feature _sounds_ useful but don't have a specific demand to implement it right now, we proceed as per the comments above (i.e. direct users to either move to prepared or use their products toolset in combination with OSEntryBeanMBean (Narayana: narayana-full 5.0.4.Final-SNAPSHOT API)) and then I will also raise a Jira (unassigned and without version number) to add the tooling we discuss above to call a new resolve() method later (and link to this discussion).

Personally, I think we should also take the opportunity to deprecate getEntityReference (narayana/HeuristicInformation.java at master · jbosstm/narayana · GitHub) and then I can also put a link in its @deprecated annotation to this new Jira. That may help to flush some votes out of the woodwork too.

Is that OK with everyone?

[marklittle]

Sounds ok as long as we update the documentation on managing heuristics. I took a look through the docs yesterday and there's nothing there that addresses this topic as far as I can tell, i.e., don't assume moving back to prepared is right, resolve out of our tool and then go in to remove reference from log etc.

We should also deprecate the entire HeuristicInformation interface.

[tomjenkinson]

I have created the following Jiras:

1. [JBTM-2328] Deprecate the HeuristicInformation interface - JBoss Issue Tracker to deprecate the current HeuristicInformation stuff

2. [JBTM-2329] Add a management API that an AbstractRecord provider can implement to clear the heuristic state of a transac… to add the new facility for an AbstractRecord implementor to provide a way to clear a heuristic

3. [JBTM-2330] Document the procedure to discover and resolve heuristically completed branches in Narayana - JBoss Issue Tr… to document the way to resolve heuristics

I have linked them all to this discussion.