Introduction

In this wiki page we'll look at how JBossTS recovery works within JBossAS. We'll spend a bit of time looking at the example code too, though as usual the manuals are the best source of material on the what's and why's of recovery. This article applies to JBossAS 4.2 and EAP versions based on is (EAP 4.2, 4.3)

JBoss Messaging recovery

JBM provides its own XAResources and instructions on how to recover them. See http://www.jboss.org/file-access/default/members/jbossmessaging/freezone/docs/userguide-1.4.0.SP3/html_single/index.html#recovery

JBossJCA <xa-datasource> recovery

XA transaction aware database connection pools configured via -ds.xml files rely upon the JBossJCA XAResources. Approaches to recovery for these include using either bypassing the JCA and recovering using the standard JDBC recovery modules from JBossTS standalone, or using the AppServerJDBCXARecovery module. For the former see the JBossTS documentation and the JDBC section of the JBossTSRecovery wiki page . For the latter see JBTM-319 and the javadoc for AppServerJDBCXARecovery Note that AppServerJDBCXARecovery is not present in JBossAS (you need to download and build it from source) or early EAP releases. It is in EAP 4.2.CP03, EAP 4.3.CP01 and later EAP releases.  AppServerJDBCXARecovery does not currently support invoker security, see JBTM-432

For recovery of custom XAResource types not covered in the sections above, keep reading.

JTA recovery

Assuming you have deployed JBossTS as per the JTA

or JTS notes, you need only change your jbossjts-properties.xml to enable:

        <property             name="com.arjuna.ats.arjuna.recovery.recoveryExtension5" value="com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule"/>

You can remove the other entry as we do not need it for this example.

Clean your ObjectStore prior to running the example.

Then download and build the example. You should deploy this into the default configuration for JBossAS and run the application server instance. Once it is ready, run the example client java org.jboss.ejb.TestClient. Within the JBossAS log output, you will eventually see:

10:43:27,392 INFO  STDOUT enlisting resources  10:43:27,399 INFO  STDOUT ExampleResource: ExampleXAResource (Constructor)  10:43:27,401 INFO  STDOUT ExampleResource: start  10:43:27,408 INFO  STDOUT ExampleResource: ExampleXAResource (Constructor)  10:43:27,409 INFO  STDOUT ExampleResource: isSameRM  10:43:27,411 INFO  STDOUT ExampleResource: start  10:43:27,416 INFO  STDOUT committing  10:43:27,419 INFO  STDOUT ExampleResource: end  10:43:27,431 INFO  STDOUT ExampleResource: prepare  10:43:27,433 INFO  STDOUT ExampleResource: end  10:43:27,435 INFO  STDOUT ExampleResource: prepare  10:43:27,444 INFO  STDOUT ExampleResource: writeObject (Serialized)  10:43:27,448 INFO  STDOUT ExampleResource: writeObject (Serialized)  10:43:27,458 INFO  STDOUT ExampleResource: commit,xid=< 131075, 28, 26, 1--3f57fe99:ead3:44757bdb:65-3f57fe99:ead3:44757bdb:66 >,onePhase=false  10:43:27,470 INFO  STDOUT ExampleResource: Sleeping 20000 milliseconds

At this stage, we have enlisted two XAResource instances with the transaction and told both to prepare. Because both implementations are serializable, their states have been written into the transaction log for recovery purposes and the transaction has entered the second (commit) phase. At this point it blocks and you should kill your JBossAS instance.

If you then look in the ObjectStore, you should see the transaction log entry:

PutObjectStoreDirHere//HashedActionStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/4: total 8  -rw-rr   1 marklitt  marklitt  752 May 25 10:43 -3f57fe99_ead3_44757bdb_65

Now restart your JBossAS instance and wait. Eventually you will see the recovery system start to work and recognise that there is an inflight transaction. It will then re-activate that transaction and start to recover any participants (XAResources in this case) that were registered with it. The log output will look similar to:

If you look within the ObjectStore, the transaction log has been removed because both participants successfully recovered.

JTS recovery

First ensure that your JBossTS installation is configured to use the

JTS implementation of the JTA. Make sure that your jbossjts-properties.xml file is correctly configured. You need:

<property name="com.arjuna.ats.jta.jtaTMImplementation" value="com.arjuna.ats.internal.jta.transaction.jts.TransactionManagerImple"/>          <property name="com.arjuna.ats.jta.jtaUTImplementation" value="com.arjuna.ats.internal.jta.transaction.jts.UserTransactionImple"/>

and

        <property             name="com.arjuna.ats.arjuna.recovery.recoveryExtension5" value="com.arjuna.ats.internal.jta.recovery.jts.XARecoveryModule"/>

Clean your ObjectStore prior to running the example.

Next download and build the example using the build-ejb.sh script. You may have to modify this for your operating system of choice. The setup.sh is provided for additional convenience where necessary. When the build is complete, you will have a recovery-test.jar which should should deploy into your JBossAS installation (using the all configuration).

Now start your JBossAS instance. Once that is ready, we need to run the example and cause a crash. To do this, execute java org.jboss.ejb.TestClient and you will eventually see a stack trace, part of which is shown below:

Exception in thread "main" java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is:           java.io.EOFException          at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:203)          at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:126)          at org.jboss.invocation.jrmp.server.JRMPInvoker_Stub.invoke(Unknown Source)          at org.jboss.invocation.jrmp.interfaces.JRMPInvokerProxy.invoke(JRMPInvokerProxy.java:119)          at org.jboss.invocation.InvokerInterceptor.invokeInvoker(InvokerInterceptor.java:227)          at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:167)          at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)          at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:55)

Plus your JBossAS instance will have exited. This is correct behaviour: out test has just crashed the server.

If you look in the ObjectStore, you will see two state entries:

PutObjectStoreDirHere//ShadowNoFileLockStore/defaultStore/CosTransactions/XAResourceRecord: total 16  -rw-rr   1 marklitt  marklitt  1008 May 24 14:14 -3f57fe99_e758_44745bf8_85  -rw-rr   1 marklitt  marklitt  1008 May 24 14:14 -3f57fe99_e758_44745bf8_89

and if you check the server log, you'll see something like the following:

11:14:04,195 INFO  STDOUT work  11:14:04,213 INFO  STDOUT enlisting resources  11:14:04,222 INFO  STDOUT ExampleXAResource: ExampleXAResource (1, true)  11:14:04,230 INFO  STDOUT ExampleXAResource: start  11:14:04,241 INFO  STDOUT ExampleXAResource: ExampleXAResource (2, true)  11:14:04,243 INFO  STDOUT ExampleXAResource: isSameRM  11:14:04,258 INFO  STDOUT ExampleXAResource: start  11:14:04,338 INFO  STDOUT committing  11:14:04,343 INFO  STDOUT ExampleXAResource: end  11:14:04,344 INFO  STDOUT ExampleXAResource: prepare < 131072, 28, 26, 1--3f57fe99:eb33:4475834c:79-3f57fe99:eb33:4475834c:7b >  11:14:04,346 INFO  STDOUT writing XID < 131072, 28, 26, 1--3f57fe99:eb33:4475834c:79-3f57fe99:eb33:4475834c:7b >  11:14:04,405 INFO  STDOUT ExampleXAResource: end  11:14:04,406 INFO  STDOUT ExampleXAResource: prepare < 131072, 28, 26, 1--3f57fe99:eb33:4475834c:79-3f57fe99:eb33:4475834c:7f >  11:14:04,407 INFO  STDOUT writing XID < 131072, 28, 26, 1--3f57fe99:eb33:4475834c:79-3f57fe99:eb33:4475834c:7f >  11:14:04,535 INFO  STDOUT ExampleXAResource: commit,xid=< 131072, 28, 26, 1--3f57fe99:eb33:4475834c:79-3f57fe99:eb33:4475834c:7b >,onePhase=false

At this stage, we have enlisted two XAResource instances with the transaction and told both to prepare. JBossTS writes recovery information on behalf of each

instance within the ObjectStore: each instance is actually a CosTransactions::Resource implementation, which wraps the registered XAResources. At this point it blocks and you should kill your JBossAS instance.

If you look in the ObjectStore, you will see the two state entries mentioned above:

PutObjectStoreDirHere//ShadowNoFileLockStore/defaultStore/CosTransactions/XAResourceRecord: total 16  -rw-rr   1 marklitt  marklitt  1032 May 25 11:14 -3f57fe99_eb33_4475834c_7c  -rw-rr   1 marklitt  marklitt  1032 May 25 11:14 -3f57fe99_eb33_4475834c_80

as well as the transaction log entry. Each CosTransactions::Resource entry contains information on the XAResource, the transaction within which the XAResource was registered and a remote reference to an OTS::RecoveryCoordinator instance, which will be used to drive recovery.

Now restart your JBossAS instance and wait. When recovery occurs, you will see:

15:05:48,683 INFO  STDOUT ExampleXAResource: readObject (Deserialized  15:05:48,708 INFO  LoggerI18N Com.arjuna.ats.internal.jta.recovery.jts.orbspecific.commit XA recovery committing < 131072, 27, 25, 1-7f000001:ec40:4475b888:797f000001:ec40:4475b888:7b >  15:05:48,710 INFO  STDOUT ExampleXAResource: commit,xid=< 131072, 27, 25, 1-7f000001:ec40:4475b888:797f000001:ec40:4475b888:7b >,onePhase=false  15:05:48,717 INFO  STDOUT ExampleXAResource: readObject (Deserialized  15:05:48,781 INFO  STDOUT ExampleXAResource: commit,xid=< 131072, 27, 25, 1-7f000001:ec40:4475b888:797f000001:ec40:4475b888:7f >,onePhase=false  15:05:48,789 INFO  LoggerI18N Com.arjuna.ats.internal.jta.recovery.jts.orbspecific.commit XA recovery committing < 131072, 27, 25, 1-7f000001:ec40:4475b888:797f000001:ec40:4475b888:7f >

If you check the ObjectStore, you will see that both CosTransactions::Resource entries have gone, along with the transaction log. Recovery has completed.