Failure on XAResourceRecoveryTest

[clebert.suconic]

I need some help understanding this...

XAResourceRecoveryTest::testRecoveryOnSend will aways fail if running after XARecoveryTest::testMockCoordinatorRecoveryWithJBossTSXids.

After some investigation, XAResourceRecoveryTest would fail by simply initializing TxControl by calling any method (such as TxControl.isEnabled()).

http://anonsvn.jboss.org/repos/messaging/trunk/tests/src/org/jboss/test/messaging/jms/XARecoveryTest.java

This is because testMockCoordinatorRecoveryWithJBossTSXids is instantiating com.arjuna.ats.jta.xa.XidImple, and that is initializing TxControl which is starting a ObjectStore.

Later on testRecoveryOnSend, the test would get a message "Told not to rollback {0}" on logs. I still don't know what's the reason but any help is welcome.

I was considering to comment the Mock Test out until we find a way to use the XidImple directly.

[marklittle]

Hi Clebert. I'll try to take a stab at this, but I think I need more information from you.

I need some help understanding this...

XAResourceRecoveryTest::testRecoveryOnSend will aways fail if running after XARecoveryTest::testMockCoordinatorRecoveryWithJBossTSXids.

Fails in what way? Can you c&p the precise error message you get?

After some investigation, XAResourceRecoveryTest would fail by simply initializing TxControl by calling any method (such as TxControl.isEnabled()).

That should be OK. This class is just a wrapper/helper for things that are idempotent.

http://anonsvn.jboss.org/repos/messaging/trunk/tests/src/org/jboss/test/messaging/jms/XARecoveryTest.java

This is because testMockCoordinatorRecoveryWithJBossTSXids is instantiating com.arjuna.ats.jta.xa.XidImple, and that is initializing TxControl which is starting a ObjectStore.

No, it doesn't start an objectstore (an objectstore can never be started). It assigns one to the instance variable if one has not already been created, but that's it and it should be idempotent, i.e., subsequent tests would just do the same thing if they had to start from scratch.

Later on testRecoveryOnSend, the test would get a message "Told not to rollback {0}" on logs. I still don't know what's the reason but any help is welcome.

This message is coming from recovery and is because the recovery module has found some Xids that it doesn't know about and rather than roll it back (which would be bad if the machine/objectstore is being shared between other TM instances) it puts out this warning and ignores it. Check the TS failure docs on NodeName. If you can't find it, ping me on IM.

I was considering to comment the Mock Test out until we find a way to use the XidImple directly.

Hopefully that won't be necessary.

[clebert.suconic]

The test fails on an assertion...

It's rolling back instead of doing a commit.


It's like initializing TxControl by hand is making Recovery "to think" that there is another Xid on the storage.

[marklittle]

Once TxControl has been initialized, it shouldn't affect any other tests. The ObjectStore contains logs on behalf of many different transactions.

Your test infrastructure doesn't delete the ObjectStore contents by any chance? TS won't do that, so we're safe there. But is there a chance something else is?

You can ls the contents of the ObjectStore at the end of each test. That might help to determine what's left there to recover (or not).

[clebert.suconic]

We play with starting/stopping the TransactionManager during our testsuite...

And we use different ObjectStores for each time we start another TransactionManager, by doing:

System.setProperty(com.arjuna.ats.arjuna.common.Environment.OBJECTSTORE_DIR, newObjectStore + new GUID().toString());

I have already tried to create XidImple only after starting the TransactionManager and I still get a failure....

I will investigate this.


Thanks Mark!

[marklittle]

OK, the problem there might be that the recovery manager isn't looking at the right object store for the log? Why not clear out the existing ObjectStore and just re-use it in the next test? That's what our JBossTS QA does.

[clebert.suconic]

Another problem that I found was, when initializing TxControl, that is also initializing the StatusManager in a socket, and that caused the RecoverModule not being able to connect somehow. I'm still investigating.

[clebert.suconic]

Problem solved!

Our testsuite was setting both ObjectStoreDir and XA_NODE_IDENTIFIER on every test.

This wasn't a problem until we used XidImple directly, as that started a StatusManager before we set XA_NODE_IDENTIFIER, and later the subsequent test would fail on getTransactionStatus:

Because of this scenario, process_id would never be found on _tscTable:

From jbossts (arjuna)... com.arjuna.ats.arjuna.recovery.TransactionStatusConnectionManager:

public int getTransactionStatus(String transactionType, Uid tranUid)
 {
 int status = ActionStatus.INVALID;

 // extract process id from uid, use to index into
 // hash table to obtain transaction status connector
 // with which to retrieve the transaction status.

 String process_id = get_process_id(tranUid);

 if (!_tscTable.containsKey(process_id))
 {
 updateTSMI();
 }

 if (_tscTable.containsKey(process_id))
 {
 TransactionStatusConnector tsc = (TransactionStatusConnector) _tscTable.get(process_id);

 if (tsc.isDead())
 {
 _tscTable.remove(process_id);
 tsc.delete();
 tsc = null;
 } else
 {
 status = tsc.getTransactionStatus(transactionType, tranUid);
 }
 }

TGIF and this is now solved!

And thanks for your help Mark

[marklittle]

Ah, OK. Glad it was sorted out.