JCA Adapter & Recovery

[maxant]

I have written my own resource adapter and it is bound into XA transactions perfectly.  during testing I want to see if the transaction manager can recover an unfinished transaction, so during the call to the XAResource#commit(Xid, boolean) method I kill the JVM.

When JBoss AS is started again, I get the following two lines of log every ~2 minutes, but the transactions are recovered.  I was expecting the commit method to be called again.

20:24:54,360 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a10be5e:5ac60acf:51cc48ab:1f, node_name=1, branch_uid=0:ffff0a10be5e:5ac60acf:51cc48ab:21, subordinatenodename=null, eis_name=java:/eis/SAPResourceAdapter >, heuristic: TwoPhaseOutcome.FINISH_OK, product: SAP Resource Adapter/1.0, jndiName: java:/eis/SAPResourceAdapter com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@8bd6e74 >

20:24:55,629 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a10be5e:5ac60acf:51cc48ab:1f, node_name=1, branch_uid=0:ffff0a10be5e:5ac60acf:51cc48ab:21, subordinatenodename=null, eis_name=java:/eis/SAPResourceAdapter >

In the ironjacamar docs I read that I need to implement org.jboss.jca.core.spi.recovery.RecoveryPlugin, so I made my ResourceAdapter do that, and added it to ironjacamar.xml, which is in the META-INF folder of my Resource Archive, which is inside an EAR.

<?xml version="1.0" encoding="UTF-8"?>

<ironjacamar>

    <connection-definitions>

        <connection-definition

            class-name="ch.maxant.jca_demo.sapresourceadapter.SAPManagedConnectionFactory"

            jndi-name="java:/eis/SAPResourceAdapter">

            <recovery>

                <recover-plugin class-name="ch.maxant.jca_demo.sapresourceadapter.SAPResourceAdapter" />

            </recovery>

        </connection-definition>

    </connection-definitions>

</ironjacamar>

What do I have to do, in order for the transaction manager to call my resource adapter to commit the unfinished transaction?

I'm using JBoss AS 7.1.0.Final

Thanks,

Ant

[tomjenkinson]

I have moved this discussion to the IronJacamar forum as it seems more related to that that transactions at the moment and you will probably get a more accurate response here.

[jesper.pedersen]

First of all - use WildFly 8.0.0.Alpha2.

You only need a recovery plugin if there is special recovery semantics for the resource adapter - look at the IronJacamar source code to get an idea of existing implementations.

[maxant]

If I don't provide a recovery plugin, my code still isn't called after the server is restarted.  I was expecting the container to call the recover method on my XAResource implementation, but it doesn't do that.  What could be the problem?

[maxant]

@Jesper: I have switched to Wildfly 8.0.0.Alpha2

I set a breakpoint in my JCA Adapters XA Resource "commit" method, which is called after the prepare method.  I crash the server during the commit so that I can test what happens when the server restarts.  The server recognises that there is still a transaction to complete, but it just logs the following, similar to before:

22:18:33,199 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could

not find new XAResource to use for recovering non-serializable XAResource XAReso

urceRecord < resource:null, txid:< formatId=131077, gtrid_length=29, bqual_lengt

h=36, tx_uid=0:ffffc0a80127:-2363921:51d5d802:d, node_name=1, branch_uid=0:ffffc

0a80127:-2363921:51d5d802:13, subordinatenodename=null, eis_name=java:/eis/SAPRe

sourceAdapter >, heuristic: TwoPhaseOutcome.FINISH_OK, product: SAP Resource Ada

pter/1.0, jndiName: java:/eis/SAPResourceAdapter com.arjuna.ats.internal.jta.res

ources.arjunacore.XAResourceRecord@117c7fd >

22:18:36,503 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could

not find new XAResource to use for recovering non-serializable XAResource XAReso

urceRecord < resource:null, txid:< formatId=131077, gtrid_length=29, bqual_lengt

h=36, tx_uid=0:ffffc0a80127:-2363921:51d5d802:d, node_name=1, branch_uid=0:ffffc

0a80127:-2363921:51d5d802:16, subordinatenodename=null, eis_name=unknown eis nam

e >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.

arjunacore.XAResourceRecord@19cda3d >

22:18:40,607 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAR

esource to recover < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0

:ffffc0a80127:-2363921:51d5d802:d, node_name=1, branch_uid=0:ffffc0a80127:-23639

21:51d5d802:13, subordinatenodename=null, eis_name=java:/eis/SAPResourceAdapter

>

22:18:43,384 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAR

esource to recover < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0

:ffffc0a80127:-2363921:51d5d802:d, node_name=1, branch_uid=0:ffffc0a80127:-23639

21:51d5d802:16, subordinatenodename=null, eis_name=unknown eis name >

Do I need to make any of the classes serializable?  Since it says: "Could not find new XAResource to use for recovering ***non-serializable*** XAResource"?

[tomjenkinson]

Hi Ant,

The serializable message most likely does not apply to your scenario.

The breakpoint to set is in your XAResource::recover method, make sure that is getting called.

Tom

[maxant]

Hi Tom,

It isn't being called.  Why could that be?

The class is shown below - altho the recover method is returning null at the moment, it isn't printing the log, or if I use the debugger, it doesn't stop in that method.

-----------------------

package ch.maxant.jca_demo.sapresourceadapter;

import java.io.Serializable;

import java.util.logging.Level;

import java.util.logging.Logger;

import javax.transaction.xa.XAException;

import javax.transaction.xa.XAResource;

import javax.transaction.xa.Xid;

import org.jboss.security.Base64Utils;

public class XASAPResource

  implements XAResource, Serializable

{

  private final Logger log = Logger.getLogger(getClass().getName());

  private SAPManagedConnection conn;

  private int timeout = 300;

  public XASAPResource(SAPManagedConnection conn) {

    this.conn = conn;

  }

  public void commit(Xid xid, boolean onePhase)

    throws XAException

  {

    this.log.log(Level.FINE, "COMMIT " + onePhase + "/" + gtid(xid));

    try

    {

      this.conn.webService().confirm(gtid(xid));

      this.conn.cleanup();

    }

    catch (Exception e)

    {

      this.log.log(Level.SEVERE, "Failed to CONFIRM", e);

      throw new XAException(-3);

    }

  }

  public void end(Xid xid, int flags)

    throws XAException

  {

    String s = "-";

    if (flags == 33554432)

      s = "TMSUSPEND";

    else if (flags == 536870912)

      s = "TMFAIL";

    else if (flags == 67108864) {

      s = "TMSUCCESS";

    }

    this.log.log(Level.INFO, "END flags=" + s + "(" + flags + ")" + "/" + gtid(xid));

  }

  public void forget(Xid xid)

    throws XAException

  {

    this.log.log(Level.INFO, "FORGET " + gtid(xid));

  }

  public int getTransactionTimeout()

    throws XAException

  {

    return this.timeout;

  }

  public boolean isSameRM(XAResource xares)

    throws XAException

  {

    this.log.log(Level.INFO, "isSameRM " + xares);

    return equals(xares);

  }

  public int prepare(Xid xid)

    throws XAException

  {

    this.log.log(Level.INFO, "PREPARE " + gtid(xid));

    if (!this.conn.wasTrySuccessful().booleanValue()) {

      throw new XAException(100);

    }

    return 0;

  }

  public Xid[] recover(int arg0)

    throws XAException

  {

    this.log.log(Level.INFO, "RECOVER " + arg0);

    return null;

  }

  public void rollback(Xid xid)

    throws XAException

  {

    this.log.log(Level.INFO, "ROLLBACK " + gtid(xid));

    try

    {

      this.conn.webService().cancel(gtid(xid));

      this.conn.cleanup();

    }

    catch (Exception e)

    {

      throw new XAException(-3);

    }

  }

  public boolean setTransactionTimeout(int timeout)

    throws XAException

  {

    this.log.log(Level.INFO, "SET TRANSACTION TIMEOUT " + timeout);

    this.timeout = timeout;

    return true;

  }

  public void start(Xid xid, int arg1)

    throws XAException

  {

    this.log.log(Level.INFO, "START " + arg1 + "/" + gtid(xid));

    this.conn.setCurrentTxId(gtid(xid));

  }

  private String gtid(Xid xid)

  {

    return Base64Utils.tob64(xid.getGlobalTransactionId());

  }

}

[tomjenkinson]

Hi Ant,

It looks to me like: com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule::addXAResourceRecoveryHelper(XAResourceRecoveryHelper) isn't being called for your adapter. You could try a breakpoint in there to confirm that, depending on the version of IJ you are using the source for JBoss Transactions is available https://github.com/jbosstm/narayana/ or http://anonsvn.jboss.org/repos/labs/labs/jbosstm/tags/.

IronJacamar will call addXAResourceRecoveryHelper for an appropriately defined adapter so I presume there is an issue in your adapter?

Thanks,

Tom

[jesper.pedersen]

IronJacamar operates on the JBoss Transaction SPI, which doesn't contain these methods.

Hence the Narayana wiring for https://github.com/jbosstm/jboss-transaction-spi/blob/master/src/main/java/org/jboss/tm/XAResourceRecoveryRegistry.java must take care of that.

Tom, can you double check if that happens.

[tomjenkinson]

My apologies, I wasn't being entirely accurate in paragraph 2 of my response:

sed

> IronJacamar will call addXAResourceRecoveryHelper for an appropriately defined adapter so I presume there is an issue in your adapter?

to:

> IronJacamar will, for an appropriately defined adapter, call addXAResourceRecovery on our XAResourceRecoveryRegistry implementation which results in a call to the internal JBoss Transaction addXAResourceRecoveryHelper method which I suggested adding a break point for above. I therefore presume there is an issue in your adapter?

Tom

[maxant]

Hi Tom & Jesper,

The method XARecoveryModule#addXAResourceRecoveryHelper gets called with:

- a wrapper around a org.jboss.as.ejb3.remote.EJBTransactionRecoveryService@c72d50

- a wrapper around a org.jboss.jca.core.tx.jbossts.XAResourceRecoveryImpl@1e53677 (my datasource)

The EJB Transaction Recovery Service is passed into the method because the ServiceControllerImpl starts the EJBTransactionRecoveryService which calls the RecoveryManagerService to add the XAResourceRecovery.

Similarly, the Datasource is added because the ServiceControllerImpl starts the XaDatasourceService which adds the Datasource to the RecoveryManagerService.

Which Service should be calling my Resource Adapter, and why isn't it doing that?

Once the recovery management is setup, JBoss goes on to create a connection factory using my adapter, and then starts my resource adapter.  It looks as though the resource adapter is started after everything has been added to the recovery module.

I've posted all the code to https://github.com/maxant/share/tree/master/resourceadapterdemo so that you can take a look if required.

I have no deployment descriptors except for the ironjacamar.xml that is posted at the start of this thread.  The only two annotations that I have are:

@Connector(reauthenticationSupport = false, transactionSupport = XATransaction)

    on the ResourceAdapter implementation, and:

@ConnectionDefinition(connectionFactory = classOf[SAPConnectionFactory],

    connectionFactoryImpl = classOf[SAPConnectionFactoryImpl],

    connection = classOf[SAPConnection],

    connectionImpl = classOf[SAPConnectionImpl])

    on the ManagedConnectionFactory implementation.

Am I missing something?

(sorry, some of the code is in Scala, but that shouldn't be the reason it isn't working)

[jesper.pedersen]

Look at the TRACE log from org.jboss.jca and org.jboss.as.connector in order to see how your resource adapter is activated. It needs to be identified with <transaction-support> of XATransaction in order for recovery to be enabled.

Where is your XASAPResource used ?

[maxant]

Hmmm, I posted a wrong project at github - I've corrected it now.

The XA Resource is at https://github.com/maxant/share/blob/master/resourceadapterdemo/SAPConnector/connectorModule/ch/maxant/jca_demo/sapresourceadapter/XASAPResource.java

The XASAPResource isn't explicitly used, rather line 44 of https://github.com/maxant/share/blob/master/resourceadapterdemo/SAPTestClient/src/ch/maxant/jca_demo/client/SomeServiceThatCallsSAP.java creates an SAPConnection from the injected factory.  JBoss creates the XASAPResource by calling the getXAResource method of the SAPManagedConnection class.  Logs show that this seems to work correctly, at least the way that I expected it to.

[jesper.pedersen]

Did you read chapter 6.5.3.5 in the JCA spec ?

[maxant]

Yes, the adapter conforms to section 6.5.3.5 of the JCA spec.  See https://github.com/maxant/share/blob/master/resourceadapterdemo/SAPConnector/connectorModule/ch/maxant/jca_demo/sapresourceadapter/SAPManagedConnection.scala and lines 67-71 of https://github.com/maxant/share/blob/master/resourceadapterdemo/SAPConnector/connectorModule/ch/maxant/jca_demo/sapresourceadapter/SAPManagedConnectionFactory.scala

During deployment it logs (near the end of the line) that transaction support is XATransaction (presubably taken from the annotation):

22:04:18,448 DEBUG [org.jboss.as.connector.deployment] (MSC service thread 1-5) ParsedRaDeploymentProcessor: CMD=<?xml version="1.0" encoding="UTF-8"?><connector version="1.6" metadata-complete="false"><module-name></module-name><vendor-name></vendor-name><eis-type></eis-type><license><license-required>false</license-required></license><resourceadapter><resourceadapter-class>ch.maxant.jca_demo.sapresourceadapter.SAPResourceAdapter</resourceadapter-class><config-property><config-property-name>url</config-property-name><config-property-type>java.lang.String</config-property-type><config-property-value>http://localhost:8080/SAPService</config-property-value><config-property-ignore>false</config-property-ignore><config-property-supports-dynamic-updates>true</config-property-supports-dynamic-updates><config-property-confidential>false</config-property-confidential></config-property><outbound-resourceadapter><connection-definition><managedconnectionfactory-class>ch.maxant.jca_demo.sapresourceadapter.SAPManagedConnectionFactory</managedconnectionfactory-class><connectionfactory-interface>ch.maxant.jca_demo.sapresourceadapter.SAPConnectionFactory</connectionfactory-interface><connectionfactory-impl-class>ch.maxant.jca_demo.sapresourceadapter.SAPConnectionFactoryImpl</connectionfactory-impl-class><connection-interface>ch.maxant.jca_demo.sapresourceadapter.SAPConnection</connection-interface><connection-impl-class>ch.maxant.jca_demo.sapresourceadapter.SAPConnectionImpl</connection-impl-class></connection-definition><transaction-support>XATransaction</transaction-support><reauthentication-support>false</reauthentication-support></outbound-resourceadapter><inbound-resourceadapter><messageadapter></messageadapter></inbound-resourceadapter></resourceadapter></connector>

22:04:18,551 TRACE [org.jboss.as.connector.deployers.RADeployer] (MSC service thread 1-5) ResourceAdapter defined in classloader: ModuleClassLoader for Module "deployment.SAPTest.ear.SAPConnector.rar:main" from Service Module Loader

Then a little later it parses the ironjacamar.xml, where there is a recovery tag (containing a recover-plugin, that I was trying out):

22:04:18,557 DEBUG [org.jboss.as.connector.deployers.RADeployer] (MSC service thread 1-5) Activating: <connection-definition class-name="ch.maxant.jca_demo.sapresourceadapter.SAPManagedConnectionFactory" jndi-name="java:/eis/SAPResourceAdapter" enabled="true" use-java-context="true" use-ccm="true"><recovery no-recovery="false"><recover-plugin class-name="ch.maxant.jca_demo.sapresourceadapter.SAPResourceAdapter"></recover-plugin></recovery></connection-definition>

Then I get a warning from the RADeployer that there is a missing recovery element:

22:04:18,563 TRACE [org.jboss.as.connector.deployers.RADeployer] (MSC service thread 1-5) ConnectionFactory defined in classloader: ModuleClassLoader for Module "deployment.SAPTest.ear.SAPConnector.rar:main" from Service Module Loader

22:04:18,563 INFO  [org.jboss.as.connector.deployment] (MSC service thread 1-5) JBAS010406: Registered connection factory java:/eis/SAPResourceAdapter

22:04:18,587 WARN  [org.jboss.as.connector.deployers.RADeployer] (MSC service thread 1-5) IJ020016: Missing <recovery> element. XA recovery disabled for: java:/eis/SAPResourceAdapter

It goes on to successfully deploy the adapter, so I can use it, but clearly, XA recovery is disabled.  I searched the sources and class files that I have but couldn't find the IJ020016 message.  I also couldn't find a recovery tag in the ra.xml schema definition.  Which recovery tag is it referring to?  The ironjacamar.xml file contains a recovery tag - is it correct?

Thanks,

Ant