I am attempting to test 2PC transaction recovery between two databases using XA datasources, but I am getting varying results and transactions do not seem to recover completely leaving my databases out of sync with each other.

For testing purposes, I run a load of requests against the application and then kill the network connection to the second database.  This causes some in-doubt transaction on that database, which I can see by querying for them.  To test the recovery, I then reconnect the network and periodically check the in-doubt transaction on the database.  Sometimes they all clear, sometimes a few in-doubt transactions are left.  In either case, when I run data integrity checks against the databases I see that data is out of sync between the two, so it appears that the databases weren't recovered properly.

When I look at the applicaiton server log, I see entries like the following;

19:33:54,613 WARN  [com.arjuna.ats.arjuna] (RequestProcessor-5855) ARJUNA012266: ShadowingStore::rem

ove_state() - unlink failed on C:\jboss\standalone\data\tx-object-store\ShadowNoFileLockStor

e\defaultStore\CosTransactions\XAResourceRecord\0_ffffc0a85887_4194946_506a241c_2ea1

19:33:54,613 WARN  [com.arjuna.ats.arjuna] (Periodic Recovery) ARJUNA012226: FileSystemStore.removeF

romCache - no entry for C:\jboss\standalone\data\tx-object-store\ShadowNoFileLockStore\defau

ltStore\CosTransactions\XAResourceRecord\0_ffffc0a85887_4194946_506a241c_2ec6

19:33:54,628 WARN  [com.arjuna.ats.arjuna] (Periodic Recovery) ARJUNA012226: FileSystemStore.removeF

romCache - no entry for C:\jboss\standalone\data\tx-object-store\ShadowNoFileLockStore\defau

ltStore\CosTransactions\XAResourceRecord\0_ffffc0a85887_4194946_506a241c_2e85

19:33:54,628 WARN  [com.arjuna.ats.arjuna] (RequestProcessor-5855) ARJUNA012266: ShadowingStore::rem

ove_state() - unlink failed on C:\jboss\standalone\data\tx-object-store\ShadowNoFileLockStor

e\defaultStore\CosTransactions\XAResourceRecord\0_ffffc0a85887_4194946_506a241c_2e7c

19:33:54,644 WARN  [com.arjuna.ats.arjuna] (RequestProcessor-5855) ARJUNA012266: ShadowingStore::rem

ove_state() - unlink failed on C:\jboss\standalone\data\tx-object-store\ShadowNoFileLockStor

e\defaultStore\CosTransactions\XAResourceRecord\0_ffffc0a85887_4194946_506a241c_2eb7

I am new to JBoss, so it could be my configuration isn't correct, but I have searched trying to find the proper configuration for transaction recovery.

The datasource definition snippet of my standalone-full config file is attached.

Any help is appreciated.